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II. REQUIREMENTS DEFINITION 



Technical Approach 



The objectives of the Initial phase of this effort were; (a) to 
gather Information related to the present Air Force OJT system^ (b) to 
analyze the requirements for Improving the effectiveness of the OJT 
system* and (c) to provide an Initial delineation of alternative ways In 
which Its could support OJT requirements. The approach to achieving 
these three objectives Is outlined In the following paragraphs. 

Background Information Gathering 

Current information and policy related to Air Force OJT was 
obtained by contractor personnel through a review of the Air Force 
documents and directives and through attendance at .workshops and 
meetings conducted by Air Force OJT personnel. 

Study of Literature Sources and Relevant Directives 

Using the facilities of the Defense Technical Information Center* 
computer searches were performed and bibliographies obtained of 
literature In such areas as management Information systems* job training 
and analysis* computer aided Instruction* management planning and 
control* and configuration management. Documents that appeared to be 
appropriate for the system definition effort were obtained and reviewed. 

Field Visits to Air Force Bases and Agencies 

To gain a first-hand understanding of Air Force perceptions on OJT 
and an automated system to support training at the base level* discus- 
sions were held with personnel of the Air foTce agencies shown In 
Table I. 



TABLE I. 



AIR FORCE AGENCIES/ORGANIZATIONS VISITED 



AFMPC/MPCD 
AFMPC/MPCR 



HQ USAF/MPPT 



HQ USAF/LEY 
ANGSC/TET 



AFDSDC 



afmpc/mpcrtt 
afomc/omy 



AFLMC 



ECI 



AFMEA 



3785 FLDTG 



afhrl/id 



3700 TTW 
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visits were also made to 10 Air Force bases, where Interviews were 
conducted with base personnel concerned with OJT and other base-^level 
training programs* Types of personnel Interviewed at these bases, and 
at three MAJCOM Headquarters, are Indicated In Table 2* 

Design for Base Interviews 

The basis for the design of the structured Interviews conducted at 
the bases visited was an Initial Illustrative model of ITS provided by 
the Air Force* In this model, five ITS subsystems were Identified, with 
their corresponding components, as shown In Figure 1* A set of ques* 
tlons was developed for each subsystem to stimulate discussion of the 
concepts related to the components within the subsystem and obtain per-^ 
captions on specific aspects of each component* Many questions were 
designed for specific levels of positions within the OJT system; a key 
to these levels was provided for each question* Two additional sets of 
questions were used, one to gain Information on work center-unique OJT 
policy on such topics as upgrading In skill-level and task proficiency 
certification, and the other to determine AFS-epeclflc training prob-- 
lems * 

As the field visits progressed, the questionnaires were revised to 
delete those Items which elicited standard responses, or no response at 
all, and to add a few questions on topics which arose more or less con- 
sistently during earlier Interviews* Final Interview protocol forms 
used are shown In Appendix A* 

Definition of Requirements 

The requirements for an effective OJT system are defined In the 
portion of this section entitled "Definition of Requirements for OJT," 
and are listed In tenns of priority or Importance for the ITS 
development effort* These requirements were produced as a result of 
examining factors In which the present Air Force OJT system Is deficient 
In light of the Information gained during the field visits* The factors 
selected, and a rationale for their selection, are also described* 

Determination of ITS Performance Parameters 

It had been Initially planned to establish performance parameters 
for each of the requirements that would be Identified during Phase I of 
the project* The parameters would consist of statements of the extent 
to which the ITS must meet the criteria for an effective OJT system. 
Because virtually all of the requirements developed represented depar- 
tures from the capabilities of the present system, substantive Informa' 
tlon that could be used as a baseline for establishing the parameters 
could not be obtained from the field visits. However, an attempt was 
made to Identify some of the parametric categories that the Air Force 
should consider prior to the development, demonstration, test, and 
evaluation of the ITS* Final performance parameters for the ITS will be 
established by the Air Force. 
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TABLE 2. IirrERVIBWS CONDUCTED DURING BASE VISITS 



BRANCH, 

HAJCCm HAJCCm CBPO UNIT SECTION. IMMEDIATE AFSES AN6 

FUNCTIONAL OJT OJT OJT SHOP SUPERVISORS TRAINING TRAINING 

BASE/KAJCOH managers staff unit COHHANDERS managers chiefs TRAINERS MANAGERS STAFF 



HQ MAC 
Scott AFB 
Bergs trom 
HQ SAC 
Offutt AFB 
HQ TAC 
Langley AFB 
Andrews AFB 
Klrtland AFB 
Luke AFB 
Travis AFB 
Blythevllle AFB 
Little Rock AFB 



X 
X 



X 
X 
X 
X 
X 



X 
X 
X 
X 
X 



X 
X 



X 
X 
X 
X 
X 



X 
X 



X 
X 
X 
X 
X 



X 
X 



X 
X 
X 
X 
X 



X 
X 
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SUPPORT PUHCTIOUS 



1«AIMI14G OEVELOPHEHT 
AMD OCLIVBIY SUBSYSTEM 



TAMk proficieitcy 
objectives 

Job rctAtion 
objactLy«a 

Interface^ 

Interface^ STSs 

Ttaxrtiitg aids 

tAtk by position datA 
LMse 



TRAIMBS HAHAGEHQIT 
SUBSVSTBt 

Cntry^ did^t^oftls of 
need 

Prescription and 
schedu I i itg 

Iitdivldudl trairtiitg 
records 

Unit tralitiitg re^rds 

IitterfACes-CDC^ HMICS, 
PDS 

neaource alloc^tioit 



eViVLUATlON 4 TRAIMIHC 
ANALYSIS SUBSYSTOt 



DATA BASE HAHAGEHlHT 
SUBSVSTSI 



PBRSOMKEL SUBSYSTOt 



litdividudl perforvaitce CPU support 
•veluatioitt 



Coat aitd capacity 
aitalyaia 



coMuiticationa 
Ttorniitdlfl 



OrgartiSAtloital aitd 
peraoititel 
r^ui revert t9 



Trairtiitg requl repeitt* 



litterface^ KAJCOM ^nd Syatatt software 
Air Staff traiitiitg 

CPRs Applicationa pro^r^ 



IMITIAL 
COHPIGURATIOM: 
QUALITY 
COHTROL 

Iitceittive requir*MittS AMD HAHMDIOtT 

ANALYSIS 

Ttditaitioit THRUST 
r^uireiteitta 



Staulatioit^ tasKs rtot Skill level Litdicea 
found on job 

reading training 

authoring aids 



C ompu te r -baaed * 

per fortaance tea ting 

Job reading teate 

Comprehensive pro^raa 
evaluation 



Hardware 4 aoftware 
for baaewide 
ewpdns ion 



Str*aaLin«d 
organization 

in-house training 
capability 



ADVANCB) 
COHPlGURATlOm 
IMSTRUCTIOHAL 
OELIVERV 
THRUST 



AFUMC 
ECl 



AFU4C 
ATC 



AFKRL/OR 
AFOSDC 
AFSC/ESD 
AFMPC/*<PCD 



Overall 
AFKRL/ID 
AP/MPPT 
APWPC/MPCRTT 
Operational coa 



PART IC IP AX IMG 
AGEHCIES 



ands 



Figure 1. initial Illustrative Model 
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ihi* iipptuacU ^atid Lo develop ITS alternative design options was to 
Identify discrete options for Independent aspects of the system* such as 
the computer network conf Iguratlon* data base processing* system 
management functional responsibilities and trainee management support. 
This enabled configuration of the options In a manner that would allow 
consideration of a mix of options rather than three or four complete* 
but Inflexible* alternative configurations. 

Determination of ITS Demonstration Conditions 

Since the ITS will eventually be demonstrated under conditions that 
will permit evaluation of Its characteristics and effects* a set of 
base* Air Force specialty* and personnel characteristics defining the 
conditions for such a demonstration was developed. 



Results of Base Interviews 

At each base* the servicing Consolidated Ease Personnel Office 
(CBPO) OJT Unit coordinated with unit and squadron OJT managers In 
scheduling the Interviews about a week In advance of the visit. Most of 
those Interviewed were In homogeneous groups of supervisors and 
trainers* or unit OJT managers; groups ranged in size from 3 to 20 
Individuals. On a typical base visit of 3 days* six to eight groups 
were interviewed. Each group was provided with a short brief on the 
objectives of the project prior to discussions of approximately 2 hours 
In length. The personnel contacted represented 92 Air Force specialties 
in 30 career fields. The wide representation was a result of the 
variety of secondary AFSs held by the more than 300 persons interviewed. 
A complete list of the career fields represented is provided in Table 3. 
Summarisations of the information gathered during the base visits are 
presented here in four sub-sections which are related to broad function-- 
al areas of an improved model of an effective OJT system. They are (a) 
training development and delivery* (b) training resources management* 
(c) Instructional management* and ( d) training system management. 

Training Development and Delivery 

The first set of questions posed to the interview groups asked the 
respondents what could be done to reorient OJT toward mission 
requirements. The four most frequent responses are listed below. Each 
response was covered in greater detail through the remainder of the 
interview. The responses are not rank ordered* but are merely listed In 
the order In which they were most oftpii discussed with respondents 
during the interviews; 
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TABLE 3. CAREER FIELDS REPRESENTED AT INTERVIEWS 



20 


Intelligence 


47 


Vehicle Maintenance 


21 


Audiovisual 


51 


Computer Systems 


25 


Weather 


54 


Mechanical/Electrical 


27 


Command Control Systems 


55 


Structural /Pavements 




Operations 


57 


Fire Protection 


29 


Communications Operations 


60 


Transportation 


30 


Communications Electronics 


62 


Food Services 




Systems 


64 


Supply 


31 


Missile Electronic 


65 


Contracting 




Maintenance 


67 


Accounting and Finance, 


32 


Avionics Systems 




and Auditing 


34 


Training Devices 


70 


Administration 


36 


Wire Communications Systems 


73 


Personnel 




Maintenance 


74 


Morale, Welfare, and 


42 


Aircraft Systems Maintenance 




Recreation 


43 


Aircraft Maintenance 


75 


Education and Training 


44 


Missile Maintenance 


81 


Security Police 


46 


Munitions » Weapons 


90 


Medical 



Maintenance, and Explosive 



Ordnance Disposal 



1. Make the Specialty Training Standard (STS)/Job Proficiency 
Guide (JPG) more specific. 

2* Eliminate upgrade training per se, and substitute continuous 
qualification training. 

3. Separate OJT from promotion/career development. 

4* Eliminate the Career Development Courses (CDCs) as a 
requirement for OJT. 

The following specific areas were then addressed as being fundamen*- 
tal to training development and delivery functions In an effective OJT 
system. 

Task Proficiency Objectives . There was universal agreement across 
the Air Force specialties Interviewed that tasks as presently listed In 
the STS/JPG are not sufficiently detailed to provide an accurate basis 
either for developing effective training programs or for evaluating the 
proficiency of those personnel who have been trained. This problem was 
found to be mere pronounced In dual weapon system wings, such as 
C-5/C-141 or B-52/KC-*135. In those work centers, th.-^ tasks listed on 
the JPG have to apply to both weapon systems, even though the 
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Wtth respect to the development of task proficiency objfpctlves^ 
rc»Kpotnit*iits furthpr rtjtr*»c»<l that^ for lU on^tnoo nj+inMrtlt IVcUhIi'aI 

cilt'flcult to dt*rlve lor tion-^iDaintenance specialties^ The consensus was 
that the task proficiency objectives should originate with the 
supervisors. This consensus was qualified, however, by additional 
comments expressing concern over the administrative burden currently 
p aced on supervisors In the present OJT system. 

When asked about the possibility of utilising occupational survey 
data to establish task proficiency guidelines, most respondents had no 
specific opinions. Those that did express an opinion felt that the 
existing occupational survey methodology could certainly be beneficial 
In such effort, but that the data extracted would have to be more 
detailed to be of real value. 

Interface with Occupational Surveys ^ Additional questions were 
used to explore In greater detail what relationship. If any, should 
exist between ITS and occupational surveys. Approximately 90 percent of 
all those responding at the CBPO, unit OJT manager, and work center 
levels felt that occupational survey data were not now being effectively 
translated Into OJT requlreioents . Many noted the lack of feedback to 
the base level and expressed dissatisfaction with the survey data 
collection methodology. The 10 percent who expressed satisfaction with 
the occupational surveys noted that several STSs had been Improved after 
such surveys. 

The general view of respondents regarding the advantages to be 
gained from using occupational survey data to formulate OJT training 
requirements was that it would establish a useful pool of task 
information. They felt this would help eliminate guesswork wlthlu 
categories of tasks when the supervisor had to select tasks to train. 
The primary disadvantage was viewed as being the lack of sufficient 
detail of the survey data. Some respondents In the Civil Engineering 
(CE) squadrons believed that this lack of detail had erroneously 
resulted in some AFSs being combined, thereby complicating the 
qualification training process (e.g., the merging of the Refrigeration 
and Cryogenics AFSs). 

When asked If they felt that an OJT system In which positions were 
defined In terms of the specific tasks performed In those positions 
could generate a task list (similar to a job Inventory) for occupational 
surveys, 80 percent said **Yes**; 10 percent said '*Yes, but the task list 
might pick up too many Insignificant tasks'"; and 10 percent said 
''Perhaps'* or *'Maybe," with no amplifying remarks. Ninety-five percent 
of the respondents felt that such a system could be used as a validation 
tool for occupational surveys; that Is, It could provide a list of tasks 
currently being trained to compare against the list of tasks that 
personnel said they were performing In the fields 
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All respondents were In general agreement that the KAJCOM func- 
tional managers should be resp onslble for Implementing the results of 
occupational surveys Into the OJT system. 

Interface with the Specialty Training Standard (STS) . The respon-- 
ses to questions regarding the present utility of the STS for OJT use 
can be summarized as follows: 

1. The present STS has Insufficient detail or depth to be 
effective as a document on which to base task proficiency training. An 
example of this problem was the situation of a shop supervisor whose 
master JPG for training required 10 full pages of tasks on AF Form 797, 
Job Proficiency Guide Continuation Sheet. 

2. The current Proficiency Code Key was considered Inadequate to 
describe an Individual's proficiency, because It was subject to widely 
varying Individual Interpretation, Many respondents, from work center 
supervisors to MAJCOM staff personnel, supported a "go/no go" concept. 
That Is, the Individual can either perform the task completely and 
correctly or cannot perform the task* 

3* Many respondents felt that, rather than reflect all gecK^ral 
tasks applicable to an AFS as the STS does, a document which provld'^s 
guidelines for the conduct of OJT should be specific to a duty position. 

As to the functions that the STS should perform In an OJT system 
that Is designed to provide only job/task knowledge and proficiency and 
not career knowledge, some respondents felt that the STS should be used 
only as a basis for an Air Training Command Course Training Standard 
(CIS). Others felt that some effort should be made to expand the 
present STS Into a '"trainer's guide." This expansion would list tasks 
performed In an AFS on a specific weapon system. 

Perceptions were then sought concerning a more clear"cut division 
between career knowledge, as outlined In the STS and the Career Develop- 
ment Course (CDC), and job knowledge, which could be delivered using 
specific job-related Instructional materials. The universal response 
was that career knowledge and advancement should be the responsibility 
of each Individual and should not be managed by the OJT system. This 
concept was confirmed by virtually all those Interviewed. The main 
thrust of their views was that the CDC contributed little to task profi- 
ciency, since CDCs are not job-site related. 

Task-by-Posltlon Data Base . The concept of defining a position In 
terms of the tasks performed In that position appeared to be new to many 
respondents, and Information gathered from questions concerning the 
Identification of positions was meager. Two typical responses concern- 
ing the approximate number of positions within a specific area of 
responsibility were as follows: 

X' From a CBPO OJT manager: '*Flve thousand or so. One position 
for each person assigned to the base.'' 
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2. From the Ifonconunlssloned Officer In Charge (ifCOIC) of a missile 
electroalcs maintenance shop with 24 personnel assigned; ''There Is only 
one position In the shop. Each person Is trained on as many tasks as he 
or she can learn during the assignment." (There were 464 tasks listed 
on the JPG used In the shop.) 

Once the concept of a task-by-posltlon data base was established, 
the fiubjects had a frame of reference for further questioning. They 
proceeded to provide their views on where the responsibility for 
creating and maintaining such a data base should be vested. The consen-^ 
sus concerning responsibilities for the data base by level of management 
was as follows; 

1. Air Force/Air Staff - define common tasks for all AFSs. 

2. Major Command - add mission-oriented tasks by weapon system for 
each AFS. 

3. Branch Chiefs and Section/Shop Chiefs - add local unique tasks* 

Respondents felt that the responsibility for ensuring the accuracy 
and currency of Information In a task-^by-^posltlon data base should be 
established at the branch level or below. 

Generally, respondents believed that anyone charged with the 
management of OJT should be able to access the data base for Inquiries. 
With respect to updates, however, the general feeling was that they 
should be made by training management personnel, with supervisor Inputs^ 
but only on approval of higher authority. 

Since a task-by-posltlon data base could become the basis for 
generating Individual training requirements, respondents were asked for 
their opinions regarding where and when such training requirements 
should be generated. The answers fell Into two distinct categories; 

1* Aircraft maintenance OJT managers and work center supervisors 
supported the Initiation of training requirements by unit OJT managers 
during al rman check-In to the unit. 

2. Supervisors In the support functional areas and CBPO OJT 
managers felt the JPG could be Initiated In the CBPO during 
in-'processlng of each airman. 

Trainer Identification and Qualification . Although this topic was 
included in the earlier forms of the survey Instruments under a general 
question regarding trainers, subsequent discussions with maintenance 
training managers on the Strategic Air Command (SAC) staff and the NCOIC 
of the SAC 3902 Air Base Wing OJT staff highlighted the Importance of 
more positive control over the quality of OJT trainers. This point was 
specifically addressed In Chapter 5 of SAC Regulation 50-8, Aircraft 



Maintenance Training. As a result » a series of questions was 
Incorporated Into the Interviews to obtain further Information during 
the remainder of the visits. 

The nearly unanimous opinion of respondents to these additional 
questions was that trainers, whether the Immediate supervisor of the 
trainee or not » should be positively Identified In an Individual's 
training record. This attitude was coupled with the additional opinions 
of those respondents regarding the qualifications of trainers, as listed 
below; 

1* Trainers must be proficient In performing the task to be 
trained and must be capable of training that task. 

2* Trainers should receive formal training on how to train. 

3. Poor trainers should be "weeded out" of the system. 

While these were the expressed opinions of supervisors and OJT 
managers, It was a common practice In some work centers to assign newly 
upgraded 5-sklll level airmen as trainers prior to their receiving 
formal "trainer" training. This was due, In part, to the severe manning 
problems that existed In the middle grades and the backlog awaiting 
Field Training Detachment (FTD) OJT Advisory Service training. 

One problem Identified In this general area was related to the 
training of entry*-level airmen In Civil Engineering specialties. In 
some squadrons, civilian foremen were not providing job proficiency 
training for these airmen. When work order assignments were made, more 
proficient civilian workers, who could perform quickly and thus help 
keep labor costs down, were selected. This resulted In a lack of task 
training opportunity for the these airmen. 

Training Resource Management 

One previously Identified problem area was the lack of an effective 
methodology to determine both the cost of OJT and the capacity of units 
to conduct OJT. Therefore, the specific areas Investigated during 
Phase I of this project were the training aids and Instructional 
technology currently In use by the MAJCOMs and Individual bases and 
units; the availability of resources. Including the determination of 
training capability; the availability and types of cost and capacity 
data; and scheduling requirements and techniques. 

Training Aids and Instructional Technology . A wide variety of 
training aids were In use at all bases visited. They ranged from the 
traditional charts, mockups, and simulators to slide/tape presentations 
that were most often used In a student-paced mode. The Instructional 
System Development (ISD) process, as It was being applied to unit 
training, often produced only Instruction delivered In slide/tape 
format. Use of state-of-the-art Instructional technology In OJT was not 
In evidence. 
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Resource Avalldblllty . When asked to what degree equipment was 
av;itltible for proficiency tr.^1nlng and t^Rtfnf^, rofipondcintii provfdod fow 
infllgtits. Ont* precLflloii menaiirlng c<]alpm<*uL laborciLury (I'MtL) wuVk 
center was using special equipment on loan from the manufacturer to 
train personnel. For the most part, however, resources were available 
when needed for task proficiency training, since the training was 
generally ''unscheduled** In the production environment and occurred only 
when production generated an opportunity. Resources were available for 
production, but were reported as difficult to obtain for training 
purposes only. With the notable exceptions of aircraft weapons loading 
teams and the Security Police training programs, "actual equipment" 
resource availability for task proficiency evaluation was minimal. 

OJT Cost and Capacity . Respondents were asked where they believed 
the OJT program, as then structured, was cost effective and where It was 
not cost effective. Task proficiency training was believed to be cost 
effective where specific guidelines for delivery and evaluation had been 
established and enforced. There were a number of areas where the 
present OJT program was considered to be not co^t effective. Among 
these areas were; 

1. OJT documentation requirements, because they prevent OJT 
managers from performing their primary training functions* 

2. Ancillary training requirements, because they were excessive 
for an already depleted work forcet 

3* Career development tied to OJT, because of its Irrelevance to 
the actual Job and because of the administrative procedures required to 
manage It. 

4. Career Development Courses (CDCs) In technical specialties, 
because these courses do not Increase a trainee's task proflcency. In 
non-technical specialties, the value of CDCs was considered marginal. 
In theoretical and procedural areas, which did not change appreciably 
over time, CDCs were felt to be of some value. 

An additional point that was Introduced by respondents was that 
completion of a CDC was of more value In studying for the Specialty 
Knowledge Test (SKT) for promotion than for providing task knowledge. 
It was noted that some CDCs contain as many as seven volumes, and It may 
be up to 2 years before the Information would really be needed for 
promotion testing . 

Attempts to gather Information regarding the costs of OJT proved 

futile. Some respondents made suggestions concerning types of training 

costs that could be derived In their areas of responsibility. Several 
examples are given below: 

1. The costs of all computer products associated with OJT. 

2. The cost of staff assistance visits, both on- and off-base* 

17 

22 



3 . Paper costs * 



A* Salaries of trainers, trainees, and OJT managers. 

Attempts to gain Information on the training capacity of a unit or 
work center were equally unsuccessful. Supervisors reported receiving 
personnel-- apparently for training, since they would be transferred 
Inunedlately after upgrading — with no advance notice, no course of 
instruction, and no Inquiries regarding their capacity to train any 
Specific number of trainees. Generally, supervisors considered their 
training capacity to be from one to three 3-level trainees per trainer. 

Scheduling Requirements and Techniques . One point became very 
clear during the base visits; there was an enormous scheduling workload 
at the base and unit levels. A typical CBPO OJT staff could be required 
to schedule up to 900 people per month In various training events. CBPO 
OJT units generally had one person doing scheduling on a full-time 
basis. In maintenance, wing training management staffs had two or more 
full-time schedulers, who were provided some assistance In the sche- 
duling processes by the Maintenance Management Information Control 
System (MMICS) training subsystem on the base computer. However, the 
time required to load courses and trainees Into the base computer and to 
make updates was considered appreciable. 

Instructional Management 

In this functional area, an attempt was made to explore how trainee 
progress was managed within the present OJT program. Elements Included 
the diagnosis of training requirements for newly assigned airmen; the 
prescription and scheduling of required training; and the creation, 
content, and handling of training records at the Individual trainee and 
unit levels. Interfaces with the CDC process, the MMICS, and the 
Personnel Data System (PDS) were explored to determine what data 
exchanges were made. Questions were Included concerning the methods of 
Individual performance evaluation used to assess a tralnee*s proficiency 
on tasks that had been trained. 

Entry and Diagnosis of Training Requirements . The diagnosis of 
training requirements was being conducted as part of the supervisor's 
Initial evaluation, which was required by Chapter A of APR 50-23, On- 
The-Job Training. Respondents were unanimous In declaring that the 
Individual diagnosis of training needs. If automated, should occur at 
the unit level for support units, and at wing training management level 
for maintenance units, with verification provided by the appropriate 
supervisor. It was further agreed that such diagnosis should Include 
all base-level training requirements. As In other conceptual areas, 
these responses were qualified by the respondents* emphasizing that the 
administrative load of any diagnosis process on supervisors and OJT 
managers should be kept to a minimum. 
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l*roHcrl[>tlon and Scheduling . Preecrlptlone for training at the 
formal course level were relatively structured in the Strategic Air Cont- 
mand* An example was the Job Position Technical Training (JPTT) Program 
as delineated In SACR 50-8, Aircraft Maintenance Training. The JPTT 
program Included decision tables that specified which training courses 
were required, based on an Individual's primary AFSC, assigned weapon 
system, and time lapse versus experience factor. 

The Security Police training program was also Interesting In the 
area of training prescription and, as will be discussed later, evalua^ 
tlon of training. Security Police positions were task-^def Ined on a 
modified AF Form 1098, Special Task Certification/Recurring Training, 
which was referred to as a Duty Position 1098. Study materials for 
these specific tasks were provided In Educational Subject Block Indexes 
(CSBIs), that contalA detailed Information on performing the tasks. 
Trainees were thus made aware, early In the training process, of exact^ly 
what task knowledge and proficiency they would be required to 
demonstrate after training. 

Scheduling problems were found to exist Air Force^lde . Many 
courses were underutilised, and often classes that were full prior to 
the class's convening date suffered from high percentages of no-shows. 
This problem was particularly severe among aircraft maintenance 
personnel. Pressures produced by high sortie rates discouraged 
supervisors from releasing people for needed training. 

FTD OJT Advisory Service courses had large backlogs at several 
bases, because of spot shortages of Instructors and the numbers 
requiring training. One base had not had an OJT advisor for nearly 6 
months. During that time, new OJT trainers continued to be assigned. 

When asked how trainees were scheduled for multi-person (team) task 
performance training and testing, respondents acknowledged that trainees 
generally performed these tasks In only one position, with little 
opportunity to gain experience In other positions. In some consolidated 
unit training programs, time constraints frequently dictated that some 
trainees merely watch a task being performed, rather than having a 
training opportunity to perform the task themselves* 

There appeared to be no systematic procedures for providing 
feedback to trainees through proficiency testing. Aircraft maintenance 
supetvlsors usually provided feedback In some form when the trainee 
received a maintenance standards Quality Control evaluation. These 
evaluations, however, were designed primarily for assuring the quality 
of maintenance rather than the quality of training. 

In contrast with the generally loose structure of prescribing and 
scheduling training and evaluation testing fo;. task proficiency^ the 
system for scheduling and tracking CDC volume review exercises (VRBs), 
course examinations (CEs) , and review training for CDCs was highly 
structured, visible, and formalised. For example, when a CE for an 
Individual trainee was received on a base, the CBPO OJT staff normally 
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notified the unit concerned and established a 30-day testing "window." 
The unit then had to schedule the person for testing sometime within 
that 30 days. The test was monitored by CBPO personnel and the answer 
sheets were scored at the Extension Course Institute (£CI). If an 
airman failed, the supervisor established review training of 2 hours pox 
day for 30 days, and then the test was taken again. Pass/fall rates 
were tracked and reported at a variety of levels. 

Training Records . Questions about current and future requirements 
for training records were divided Into two groups: Individual Training 
Records and Unit (aggregate) Training Records. 

a. Individual Training Records . Respondents felt that a training 
record should be Initiated on completion of basic military training 
(BMT), rather than when an Individual reports to the first unit. In 
addition, respondent? said that certain pertinent data from BMT, such as 
reading scores, and Information concerning Resident Technical School 
performance, should be placed In the record. 

Little Information was obtained concerning squadron/unit require- 
ments for tracking Individual trainee progress In OJT. Most 
Interviewees responded that training management took care of tracking. 
Others quoted the requirements In aFR 50-23. When the question was 
reworded to ask how trainee progress should be tracked, about 20 percent 
suggested employing a checklist that could be used to record the number 
of times a task was attempted before successful completion. 

At the time of the field Visits for this project, tracking aids In 
use to monitor the progress of trainees toward task proficiency 
consisted primarily of the AF Form 1320, Training Chart. There was a 
provision In MHICS for an automated JPG that was being tested at a few 
bases. The Interviewers were shown sample J?G listings of tasks on 
which Individuals were qualified. A composite listing also provided the 
percentage of qualification of a trainee; that Is, how many tasks had 
been certified versus the total number of tasks to be trained. There 
was no provision for reporting how much training an Individual had 
received on a given task that had not yet been certified. 

Frequency of access to AF Form 623, On-The-Job Training Record, 
varied with the level of OJT management. Supervisors would normally 
require access to an Individual trainee's record several times a week. 
Unit OJT managers would require access to the record at least once a 
month while the person was In upgrade training, then twice a year after 
upgrade. The CBPO OJT staff would require access to a record a minimum 
of twice a year, once on staff assistance visits and once when 
processing upgrade requests. 

The time delay that would be tolerated In obtaining required 
training record data again depended on the level of OJT management. 
Generally, supervisors required the data Immediately In order to make 
training decisions. Wing training management In maintenance believed 
that a 2-hour delay would be acceptable. The CBPO and tlAJCOM 
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Ht^aclquarters OJT staffs, because their need was for long-term planning, 
coulil accept a delay of 24 houre or more. 

Although an Af Form 623, On-The-Job Training Record, need be 
maintained only for each Individual below grade E-7 (and for E-7 and E-*8 
retralnees), respondents at all levels of OJT management aad supervisors 
In all pay grades felt that the training record should be maintained 
throughout a person^s career, regardless of grade. 

b. Unit Training Records . Training records at the unit level 
consisted mostly of OJT status reports on which were recorded such 
statistics as the numbers of personnel In ovei time training. In 
excessive training, upgrading to the various skill levels, etc- There 
were few respondents who felt any real need for manually producing unit 
training statistics, primarily because they believed any requested data 
could be compiled when needed for a particular purpose and transmitted 
over the telephone. However, when they were apprised of the potential 
for producing continually updated computerised unit training records, 
they reacted favorably. The following levels of management were 
perceived to require such aggregate training data: 



1. 


MAJCOM OJT Staff 


. 


2. 


Wing Commander - 


on request only. 


3. 


Deputy Commander 


for Maintenance. 


4. 


Unit Conmander - 


broken out by work center/section. 


5. 


CBPO OJT Staff. 





Respondents further felt that statistics related to trainer 
utilization and training capacity data should be generated to help 
determine maximum training loads for a given unit, although true OJT 
capacity values might be difficult to determine. 

Interfaces with MMICS and PDS Data Automation Systems . To deter- 
mine how ITS could best Interface with the two primary automated systems 
which support training. Information was sought related to (a) current 
support of OJT, (b) existing data that were considered useful, and 
(c) Inputs being made by potential users of the ITS. The results Indi- 
cated that nearly all the training data that existed In both MMICS and 
PDS were useful to their respective users. 

Dissatisfaction was generally expressed by those who were least 
experienced with the systems, and who therefore could not take full 
advantage of what was available. One recurring complaint was the lack 
of space available for data In the MMICS training subsystem and the low 
priority assigned to training In both MMICS and PDS. Another frequent 
complaint was the lack of Interaction and data flow between MMICS and 
PDS. Because of the separation of both systems, a great deal of 
redundant recordkeeping was required. 
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In the MMICS training subsystem^ many managers saw a need for 
standardized training course codes » so that a course code a!: one base 
would be the same for an Identical course at another base. This lack of 
standardization presented a problem when trying to diagnose a newly 
assigned airman's training needs. 

Individual Performance Evaluations . Task proficiency evaluation of 
Individual trainees was considered very Important In determining the 
effectiveness of OJT. "Thlrd^-party" task evaluation (that Is, 
evaluation by someone other than the assigned trainer) was becoming more 
commonplace. But frequently the task evaluator, because of a shortage 
of experienced people, was the trainee* s supervisor. This tended to 
dilute the potential effectiveness of third-party evaluations. 

The Security Police were judged to have an effective system for 
task proficiency evaluation. Each trainee was evaluated by a 
representative of a Standardization Board within a specific period of 
time (typically, 30 to 60 days) after being certified. The trainee had 
to achieve a grade of 80 percent or better on oral, written, and 
practical tests covering each task on which certified. Failure on any 
task automatically triggered retraining on the failed task. A report of 
the results of the evaluation was forwarded to the trainee* s supervisor 
and the squadron commander. 

Outside of the Security Police and Air Traffic Controllers (who use 
Federal Aviation Admlnlstratlon^certlf led evaluation teams), task 
proficiency evaluation was considerably less formalized. Rarely was a 
squadron commander notified of evaluation results. Additionally, it 
appeared to be a universal practice that a trainee was evaluated In only 
one position of multi-person (team) tasks. No explanation was given for 
this, other than time constraints. 

The Maintenance Standards Evaluation Program (MS£P) was believed to 
have little Impact on training except In SAC, where MS£P teams had been 
chartered to Include training evaluations and training assistance to 
units If requested by unit commanders. 

Sklll^Level Indices . Originally designated as a possibility for an 
advanced version of ITS In the Air Force Illustrative model, this topic 
received so much reaction from personnel during the field visits that a 
decision was made to examine It In greater detail. The Air Force 
defines the 5-sklll level as being the skilled or journeyman level. 
Ainnen at the 5-sklll level ""have, through experience and training, 
shown proficiency In their AFS and can be reasonably expected to perform 
on the job without direct supervision" (APR 35^1). In reality, this may 
not be true. It Is standard practice across the Air Force^ with a few 
notable exceptions, to upgrade Individuals to the 5-sklll level at the 
time of CDC completion. The number of months spent In training closely 
coincides t^lth the number of volumes In a CDC. Certification of 
proficiency on the tasks Identified on the JPG usually occurs soon after 
the CDC course examination Is passed. The general consenHus of ntipur- 
vlsor!», OJT managers, and trainers In the field waa that, at tU<\ time of 
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upgrading) most airmen were not fully qualified in their positions. 
Airmen in some positions were said to require 2 or more years on the job 
to achieve full qualification* 

Reports and charts examined on the field visits consistently used 
the number of S'^levels on hand to indicate a desired state of readiness* 
In actuality the "5-level" airman may be only slightly more qualified 
than a 3-level and, in some cases, less qualified. It was apparent that 
personnel were often upgraded in minimum time because of pressures, real 
or perceived, to produce qualified individuals* 

Seventy percent of managers and supervisors were in favor of 
abolishing "skill levels" altogether, suggesting a system like the 
Navy*3 where skill level is inherent in rank. When this proposition was 
put to personnel at the Air Force Manpower and Personnel Center (AFMPC) 
who are involved with personnel assignments, they responded that skill 
levels were essential for making such assignments* However, the 
perception of personnel in the field was that many assignments seemed to 
be made without regard to skill levels* 

Many respondents felt that some certification indicator could be 
used based on the numbers and types of positions held in an AFS, the 
types of tasks performed, and the levels of proficiency demonstrated* 

Eighty-'five percent of those respondents felt that, at the very 
least, skill levels should reflect proficiency only In the currently 
assigned position* For example^ an airman who attains a S^^skill level 
as a Pneudraulic System Technician on an F-111 cannot be expected ini-^ 
tially to have the same level of proficiency when transferred to a 0*^5 
wing* The 15 percent who disagreed suggested that a prefix or suffix 
could be added to the Air Force Specialty Code (AFSC) to indicate 
qualification* In effect, they advocated the use of additional AFSC 
shredouts* 

When asked what factors should be considered in defining skill 
levels, the following factors were suggested: 

1* Task proficiency and task knowledge. 

2* Experience and positions held* 

3* Task proficiency and demonstrated management/supervisory 
abilities. 

Training System Management 

On the assumption that personnel charged with managing base-level 
training must have the authority as well as the tools to make the system 
work, the interviewers probed to determine what functions OJT managers 
were performing in the present system* From this information^ the ques- 
tioning was expanded to encourage respondents to give their views on 
ways to improve the organizational structure of OJT and to increase the 
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training and management utilization of personnel In the Education and 
Training career field. 

Organizational and Personnel Requirements * The concept of a 
Consolidated Base Training Office (CBTO) was put before the respondents. 
Most agreed It would be beneficial* Their main stipulation was that It 
not be combined with the Base Education Services Office, whose function 
Is not directly related to mission requirements. Other suggestions for 
an organization to manage OJT more effectively at the base level 
Included; 

I* Place unit OJT managers administratively under the CBTO» but 
physically locate them In the unit for which they are responsible, 

2, Place an officer In the Education and Training career field In 
charge of the CBTO who would report to the Wing Commander, or the Vice 
Wing Commander In his/her capacity as the Installation Inspector General 
(IG), 

It was the view of inany of the more experienced OJT managers that 
extensive training would be required for many In the Education and 
Training career field before they could function effectively as training 
analysts and developers of effective training programs* 

Job Rotation Objectives * At the time of the field visits, little 
established policy for planned job rotation within units or work centers 
was found* With the exception of personnel In the Accounting and 
Finance, and Auditing Career Field, which has a rotation policy, many of 
those Interviewed felt job rotation was wasteful because they believed 
It meant giving up a fully qualified person for an unqualified person* 
A few supervisors recognized that carefully managed job rotation could 
Increase their overall readiness level and add to the work center* s 
capability to meet contingencies* Possible constraints on an Air Force 
job rotation policy were considered to be supervisor reluctance and the 
additional cost of training* An Increase In airman motivation was 
considered a potential benefit of such job rotation* 

Interface with MAJCOM and Air Staff Training OPRs * An effort was 
made to determine not only the Interfaces that then existed between the 
OJT program and MAJCOM and training Offices of Primary Responsibility 
(OPRs), but also the additional Information reports or products that 
would be required to enhance the effectiveness of OJT* 

Generally, the MAJCOM OJT staffs and other training OPRs felt that 
they had been kept adequately Informed of problems within the OJT 
system. The results of Interviews conducted at the unit and work center 
levels In many Instances appeared to Indicate just the opposite* 
Supervisors and unit OJT managers seemed frustrated by s perceived 
Inability to effect meaningful changes within the OJT system* 
Consequently, the upward flow of Information concerning real problems 
was somewhat limited* 
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There were Isolated Instances where, when staff assistance visit 
reports detailing training problems were subiPitted by conscientious OJT 
managers, the reports were allegedly modified by superiors to reflect 
more favorably on a unites training program. 

At the MAJCOH level, respondents were asked what additional reports 
or statistics would be desirable. Some of the more pertinent replies 
are listed below; 



1. Experience quantification (a Special Experience Identifier 
[SEI] with a clock on It), 

2. The number of career staff sergeants not In OJT* 

3. Base-level report of problems In CDC perfonnance (by AFS) . 

When asked what effectiveness measures were used by higher 
authority to evaluate a unit's training program, a few respondents 
referred to task performance and documentation accuracy. The question 
was then rephrased to ask what measures could be used to evaluate a 
unites training program. The results were disappointing. Some sugges- 
tions made were to use some sort of unit productivity analysis and to 
use the results of third-party proficiency evaluations. 

Incentives . In the present OJT system, the only Incentives awarded 
relate to upgrade of trainees based on completion of CDCs and the course 
examination grade. There were no Incentives being provided for train- 
ers. Several respondents suggested that the Airman Performance Report 
(APR) was an Incentive to gain task proficiency, but questioning In this 
area did not produce substantive support for this. Every enlisted 
person In the Air Force Is evaluated In six areas on the APR. Only one 
of those areas relates to job performance In terms of technical task 
proficiency* NCOs have a separate area for management and supervision 
which becomes more relevant as grade Increases* The evaluation In each 
area can range from 0 to 9; however, most personnel viewed an evaluation 
of S as detrimental to their prospects for advancement and to their 
careers In general* Consequently, many reporting officials have been 
reluctant to assign any evaluation grade of less than 9 for fear of 
adversely affecting the career of an otherwise average airman. 

Evaluation of Training Effectiveness In Meeting Mission Require- 
ments . In answer to questions related to training effectiveness In 
meeting mission requirements, the responses revealed two basic percep- 
tions on how training effectiveness Is evaluated. Management personnel 
In the work center Indicated that the effectiveness of OJT was generally 
measured by how well a task was performed by Individuals. On the other 
hand, training OPRs at higher levels tended to believe that training 
effectiveness was measured by how well a unit performed Its mission. 

OJT managers at the C&PO level believed that staff assistance 
visits adequately assessed the effectiveness and efficiency of OJT, but 
they were discouraged when reports of the visits were ''toned down.** 
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Additional training effectiveness meaeuree suggested were; 



1* Third-party task evaluations* 
2* CDC pass/fall rates ' 
3* Number of 5-levels* 

Statistics that were considered by respondents to be significant In 
evaluating OJT Included the nufflber of people enrolled In upgrade 
training (UGT) and the number of upgrades per month* Respondents 
acknowledged that, as UGT was then being tracked, such statistics h^d 
little relevance, but they pointed out that this was better than having 
no measures of ojt effectiveness* 



Definition of Requirements for OJT 

This section provides discussions of the conclusions reached during 
this phase of the system definition effort relative to (a) a new 
approach for Air Force OJT, (b) the factors considered In defining 
system requirements, and (c) categorization of the requirements* It 
should be noted at this point that the base visits and other work 
performed during this phase resulted In an ojT concept that Is worthy of 
serious consideration If an effort to reorient base-level training to 
mission requirements Is to succeed. This concept Is one In which the 
objectives of ^ob^slte training and the objectives of career development 
are clarified with respect to mission accomplishment. 

Within the present OJT system, career development receives more 
management support and attention than does job proficiency. This 
apparently 1& a result of the availability of study materials of a 
general nature for career development, but at the same time, little 
structure and few available materials that are specifically designed for 
proficiency qualification In a duty position of an Air Force specialty. 
The relationships depicted In Figure 2 are considered to be a valid 
basis for altering the present emphasis on use of OJT as a management 
tool for monitoring career development, to more of a mlsslon-supportlve 
qualification training management and evaluation system. 

A New Concept for Air Force OJT 

The greatest Interest shown by supervisors and trainers during the 
base visit Interviews centered around the topics related to upgrading In 
skill level. What emerged from these discussions were perceptions that 
skill levels do not provide accurate Indications of actual job 
proficiency, and that they serve only as an Indication of promotion 
readiness. It was noted that upgrades In skill level are awarded 
essentially on the basis of CDC completion and meeting time 
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requirements; that Is, seldom are alrmea not upgraded because of 
deficiencies In task performance qualifications* This, In effect, has 
oriented the Air Force OJT program primarily toward management of quail-* 
flcatlon for promotion* Virtually all supervisors Interviewed believed 
that If such promotion qualification were made the responsibility of 
each Individual airman, there would be little adverse Impact on the 
numbers promoted or on retention of flrst-term airmen. Thus, a 
redirection of the emphasis of Air Force OJT away from upgrade training 
management would do much to remove the pressures to certify airmen on 
tasks for which they have not demonstrated proficiency. More emphasis 
could then be placed on the management of Job qualification training, as 
opposed to career knowledge training, with a more substantive linkage to 
the mission requirements. 

There would be several Interesting features of a work center/ 
section Job qualification training program managed and supported by the 
ITS. The basic element of training would be the task to be trained, so 
that relevant materials could be gathered and/or developed and delivered 
In a systematic fashion. Data related to the training of, and 
qualification on, each task would provide a basis for assessing 
Individual progress toward full qualification In a position and the 
effectiveness of the unit training program In meeting mission 
requirements. Training would be an ongoing process. In that the target 
date for full position qualification would not be based on an arbitrary 
minimum or maximum number of months of service In the Air Force* 
Instead, the time required for full position qualification would depend 
upon the complexities of the tasks within the position, the aptitudes 
and previously attained skills of the Individual trainee, and records of 
training provided for the position In the past. And finally. Individual 
career development could be assessed more accurately In terms of 
qualification attained In specific positions during a career, and Job 
rotation policies for Air Force specialties could be established to 
benefit both the Air Force and the Individual airman. 

Projected automation aspects of the ITS, particularly training 
qualification records Inquiry possibilities, elicited much support from 
virtually all training technicians and most supervisors. This was 
considered to be a strong Indication of the requirement for an OJT 
documentation process as '^paperless'' as It Is possible to achieve. 
However, most of those Interviewed cautioned against generating 
time-consuming data Input procedures In a new system which would prove 
to be as much a burden In the work center as the present "paper" 
documentation and forms flow* Related ITS concepts that were well 
received were automated generation of Individual training requirement 
listings, detailed tracking of trainee progress toward task proficiency, 
and providing a comprehensive record of all training received by an 
Individual airman during a career. 

The relationship between the ITS, which would provide management, 
evaluation and other support for Air Force OJT, and the need for better 
mducigometit. ot related batie-'level ancillary tralttlng requirements) mast 
alBO be addressed* The ITS design goals focus specifically on the 
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identification of tasks to be trained, managing trainees to a high level 
of qualification In a position, and the quality control aspects of such 
training. While most ancillary training requirements do not lend 
themselves to precisely the same kind of Instructional management 
approach, the capabilities would exist In the ITS to support these 
requirements at the base level In such functional areas as tracking, 
scheduling, recordkeeping, reporting, and, where appropriate, pre- and 
post-testing evaluation procedures. To the extent possible, such 
capabilities should be provided for ancillary training In the ITS, so 
that efficiencies In meeting these types of training requirements can be 
gained. 

Factors Considered In Defining OJT System Requirements 

The difficulties which had plagued the Air Force OJT program In the 
past, and the lack of acceptance of some of the program* s philosophy and 
procedures on the part of MAJCOM functional managers, have been due In 
large part to the Inability of the system to provide evidence of Its 
effectiveness In producing job-quallf led airmen. Data have been 
plentiful to support tracking progress toward career development (CDC 
completion), but little. If any, data have been generated relative to 
the outcomes of task proficiency training. An "effective" work 
center/section training program has generally been considered to be one 
In which documentation of records was acceptably accurate and In which 
the overtime and excessive training rates were kept to a minimum. It 
was evident that quite different factors should be considered In 
defining the requirements for a more effective OJT system. These 
factors are as follows: 

o Responsiveness to mission priorities. 

o Provision for skills required only for contingency operations- 

o Assurance of proficiency following OJT* 

o Optimal utilization of training resources- 

o Cost control . 

o Management Information, 

Responsiveness to Mission Priorities . This critical factor 
addresses the need for job^dlte training to result In the capability of 
the work center/section to accomplish Its mission through the 
performance of tasks which are directly related to mission requirements. 
For this to occur, the primary emphasis In OJT must be placed on 
providing airmen assigned to positions within an Air Force specialty 
with the specific skills required for the day-to-day production of the 
work center. Emphasis on requiring career knowledge training (that Is, 
knowledge of tasks, jobs, and missions within the specialty not related 
to an airman* s present assignment) would not consider the mission 
responsiveness factor as being the key to effectiveness of OJT. Career 
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developnient could reasonably be considered to be a secondary function of 
having received qualification training leading to proficiency In a 
variety of positions within an Air Force specialty throughout an Air 
Force career, as opposed to making It a flrst*tenn upgrade training 
objective • 

Provision for Skills Required Only for Contingency Operations > 
Some skills which airmen must gain are required only during contingency 
operations » This factor was considered In defining ITS system 
requirements for OJT, since It Is probable that a large portion of the 
available production time should be devoted to such training* The need 
to consider the effectiveness of training for qualification In 
contingency operations skills Is no less Important than for skills 
required for dally production* 

Assurance of Proficiency FollowlnjB OJT * One of the major factors 
contributing to lack of support In the operational environment for OJT 
Is the absence of standardized procedures for quality control of job* 
site training. In those few functional areas where training quality 
control measures are regularly employed, such as standardization boards 
In Security Police squadrons, positive effects have been observed by 
unit training managers* It should be noted that production quality 
control (QC) functions, such as aircraft maintenance QC and the 
Maintenance Standardization Evaluation Program, do not appear to have 
observable positive effects on the conduct of training In the work 
center or on attainment of task proficiency* Requirements for a new Air 
Force OJT system should address provisions for a training quality 
control function directly concerned with the effectiveness of 
qualification training assessed through systematic task proficiency 
evaluation procedures* 

Optimal Utilization of Training Resources * This factor, also 
considered In defining system requirements for ITS In support of OJT, 
addresses several deficiencies In the present OJT program related to the 
availability and scheduling of resources required to support base-level 
training. If OJT Is to be responsive to mission requirements, equipment 
and human resources must be Identified, procured, and allocated 
considering appropriate forecasts of training loads and the relative 
Importance of the tasks to be trained to the mission* Resource 
allocation and scheduling processes should be efficient and flexible 
from the standpoint of Integrating work center production workloads with 
the training to be accomplished* 

Cost Control * The ITS requirements definition process considered 
the fact that the absence of cost data In the present OJT program Is a 
major deficiency that precludes accurate assessment of the efficiency of 
base'level training. Significant decisions concerning strategies to be 
used to meet training requirements are now necessarily made based on 
Inadequate cost Information. An effective OJT system's requirements 
should specify that cost data be gathered for use In the analysis and 
control of costs for all base*level training. 



Management Information . Traditionally, Air Force training manage- 
ment at all levels has been provided with atstiatics related to the 
upgrading-in-akill-^level process in general, focuaing apecifically on 
time in training prior to CDC completions. The detailed information 
necesaary to assess OJT^s responsiveness to misaion requirements by 
providing job qualification was another factor considered in the ITS 
requirements definition. An effective OJT system should not only 
generate task proficiency and other performance dsta on a routine basis 
as training occurs, but should be capable of reporting the data in a 
form which can be readily utilized by decisionmakers and training 
managers at various levels. 

Requireinentft for an Effective OJT System 

An effective OJT system should qualify airmen to perform at 
predetermined proficiency levels in their assigned duty positions as 
defined by the tasks performed in those positions. Since the position 
is the means through which a work center accomplishes its mission, the 
degree to which the work center meets the requirements of the mission is 
a function of the numbers of airmen who are qualified to perform the 
tasks of their positions. On-the-job training, then, to be responsive 
to this requirement for position qualification, should (a) focus on task 
proficiency and qualification training, (b) provide qualified airmen for 
each position in the Air Force, (c) be able to provide valid information 
related to its effectiveness in training airmen to fill positions, and 
(d) have a high degree of flexibility to adjust to changes in mission at 
any organizational level. 

Task proficiency training, if it is to result in full position 
qualification, should be provided at the job site. This would provide 
for the optimum utilization of existing operational resources, but more 
importantly, it would provide a measure of confidence that task 
proficiency certification following training was a meaningful event that 
occurred in the production environment. 

Job-^6ite training has been considered to be difficult to develop, 
deliver, and manage because of the pressures of production, shortages of 
qualified trainers, and lack of task training opportunity. These types 
of job-site training problems cannot be resolved until more use is made 
of Instructional System Development processes in the design of OJT 
programs. An effective OJT system ahould include provisions for 
structuring job-site training in a sequence of educationally sound steps 
leading to job qualification. Further, a trainee^s progress in 
achieving the objectives of the training should be managed on an 
instructional assignment-by-^assignment basis* Tests of both knowledge 
and proficiency should be administered to ensure the achievement of 
skills in the most effective sequence and to ensure the validity of the 
training program. The performance data that can be generated through an 
appropriate trainee management process, when compared with mission 
requirements stated in terms of the tasks that imist be performed, can 
provide measures of training effectiveness heretofore unavailable for 
Air Force OJT. 
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Many trainees, trainers, Immediate supervisors, unit OJT managers, 
and others Involved with the training process are dedicated to providing 
quality training* The award of the few positive Incentives for such 
dedication has been based In part on the formalities of documentation 
and tracking* In a more effective QJT system. Individuals who demon-* 
strate effectiveness In linking the training program more closely to 
task proficiency requirements could be Id&ntlfled, and Incentives could 
be provided based on contributions to mission capability* 

Tralnlng**related deficiencies In the readiness posture of Air Force 
organizations have been difficult to Isolate from the total set of 
logistics, organizational, environmental, job characteristics, and other 
problem areas* An Important factor related to this difficulty has been 
the Inability of the present OJT system to generate data of the kinds 
needed to assess the Impact of training on readiness* To ensure 
effectiveness, the QJT system should generate these and other 
appropriate data as a function of each unites continuous training 
activity for use In Identifying the strengths and weaknesses of the 
training program* This concept should be extended to provide an 
Interface with Air Force occupational survey activities Intended to 
provide data for making formal training relevant to jobs within 
specialty areas* 

Job-*slte training resources, which are for the most part production 
resources* have not In all cases been efficiently allocated with respect 
to crltlcallty of training to mission accomplishment* This lack of 
prioritization has been evident In the competition for scarce equipment. 
Instructional media, and qualified human resources In the production 
environment* Such a situation affects the ability of supervisors and 
training managers to forecast training loads accurately, to estimate 
training capacity, and to account for the costs of training In the 
operational setting* The OJT system should provide an effective means 
of Identifying, accounting for, and allocating training resources based 
on mission requirements and cost* 

Finally, to address requirements for the development of the Total 
Force and to provide a responsive capability to meet Trained Personnel 
Requirements (TPR), an effective OJT system should have the capability 
to collect valid training performance and qualification data that could 
be provided to the assignment, classification, promotion, and other 
personnel systems of the Air Force* 

The specific requirements for an effective OJT system defined In 
this section have been categorized Into four functional groups as 
follows: 

1* Training development and delivery « 
2* Instructional management* 
3* Training resources management - 
4* Training system management* 
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Each requirement addresses one or more of the factors discussed In 
the section on ''Factors Considered In Defining OJT System Requirements.'' 
A further categorization by training system component area has been made 
within each of the four functional categories. The order In which the 
component areas are listed within each category represents their 
estimated Importance relative to the operation and maintenance of the 
total Air Force OJT system. 

Training Development and Delivery Requirements . This category of 
system requirements Includes components that (a) provide a basis for the 
design of a work center /section training program, (b) help to ensure 
that the training developed Is relevant and responsive to the mission, 
and (c) establish a baseline for the measurement of task proficiency and 
the assessment of Job qualification. Additionally, these training 
development and delivery components are Intended to facilitate the 
Introduction of training delivery technology Into the Job^slte training 
environment as a function of Air Force ISD policy. 

Instructional Management Requirements . This category of system 
requirements establishes the need for (a) accurate diagnosis of training 
deficiencies with respect to the task proficiency requirements for a 
position, (b) managing the Individual tralnee^s progress toward job 
qualification, and (c) maintaining definitive Information concerning 
training required, training In progress, and training certified at the 
Individual trainee, work center/section, and squadron/unit levels. The 
system components that are fundamental to these functions are (a) the 
airman training record component (a comprehensive account of training 
completed and skills achieved, for each airman) and (b) the trainee 
management component, which would provide capabilities to manage all 
aspf^cts of the airman* s currently required training. The Interactions 
that take place between these components and the resultant data 
generation would provide the capability to produce meaningful 
Information for managers concerning the degree to which training meets 
mission requirements. 

Training Resources Management Requirements . The control, alloca- 
tion, and scheduling of all base-level training resources are the 
primary objective of this category of system requirements. Training 
resources management Is one of the more critical factors In determining 
not only the costs of on-the-job training but also training capacity and 
capability. Effective training resources management encompasses both 
human and non-human elements. Further, the resources which limit the 
amount and kind of training that can be provided is of serious concern 
when responsiveness to mission requirements must be evaluated and when 
forecasts of training pipeline bottlenecks must be made. Finally, to 
ensure that Job-^9lte training Is delivered with minimum Impact on unit 
production, an efficient, base*level scheduling component should be pro^ 
vlded * 

Training System Management Requirements . This category of system 
requirements directly addresses providing capabilities for maintaining 
(a) the effectiveness and efficiency of Job-alte training, and (b) the 
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accounting and control of costs associated with such training* Of major 
Importance In achieving Air Force training objectives Is the quality 
control component* Requirements for training QC must be met to ensure 
that training resources are not wasted and that all elements of the 
system function as required* Training cost control requirements address 
providing the capability to monitor and to Increase the efficiency of 
base-level trainings a prime consideration In a climate of severe 
budgetary constraints* It Is by the Information provided through this 
component that funding requirements for QJT can be Identified and 
projected as mission and training loads change* The QJT system should 
be capable of Interfacing training data with other Air Force data and 
support systems so as to enhance the effectiveness of each of the 
Interfacing systems* Key relationship requirements have been Identified 
and described In the management Information Interface component* 

The requirements that were defined for an effective QJT system are 
listed In Table 4* 
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TABLE 4. REOUIRCIlBltTS POII hH BfrBCTIvB OJT SYSTBH 



TRAEtlltiG OBvCLOPHCtlT AND OBLIvCHY 



TAtik-bv-?oeit}on P^ta Basg 

U Provide a means to eflt«blish «nd 
ttt«intain a naster taBk list con- 
taining taflka applicable to ape^ 
cialtiaa within the Air Force. 

2. Provide a neana to standardize 
the Identification of each task 
on the liaster task llat In order 
that task certification/ profit 
ciency nay be accounted for uni- 
versally throughout the Air 
Force. 

3* Provide a means to establish and 
Maintain a list of tasks on the 
maater task list assigned to 
positions in a work center/aec- 
tion that are related to mission 
accompX ishment and for which 
training la required on the job. 

4 . Provide a paans to augment the 
post tion tra ining requirements 
list with local unique taska for 
which training is required on the 
30b but which are not yet estab- 
lished in the master task list. 

Provide a means to review all 
local unique tasks within a spe- 
cial ty for commonaJ ity and for 
possible incluaion in the master 
task list. 

6 * Prov ide a means to add tasks , 
delete tasksr and make revlaions 
to tasks on the master task list 
as requirsd by changss in weapons 
sytemsr contingencies* variations 
in specialty requirements r reor- 
ganizations, etc. 

7. Provide a meana to add# delete , 
and make revisions to the posi- 
tion training requirements list 
as a result of changes in weapons 
systems, contingencies, aignifi- 
cant var iat Ions in workload* 
reorganizations # etc* 

8. Provide a basis for comparing an 
airman* s previously acquired 
skills with the task proficiency 
requirements of a position for 
the purposs of generating a diag- 
nosis of the training required 
for total qualification In the 
position. 

9* Provide a basia for comparing an 
airman's previously acquirsd 
skills with the task proficiency 
requirements of a position for 
the purpose of allowiivg position 
reassignment and job ro tation 
decisions. 



10« Provlds s basis for ths snslysls 
of Air Fores occupstlonsl sctlvl- 
tles« 

11. Provide a basis for the analysis 
of job training activities within 
Air FOrcs special it iss for the 
snhancamsnt of Specialty Training 
Standards (STSs) « 

B. Task Proficiency Oblsctlves 

1. Provide a means to idsntlfy and 
def ina , in specif Ic terms , the 
bshaviors, conditions, and stan- 
dards related to ssoh task to be 
tra Ined . 

2m Provide a msans to determine if 
sn airman is qualifiea to perform 
a task in his/her assigned duty 
posit Icn after task proficiency 
training On that task. 

C- Instructional Technology 

1. Provide a means by which job-site 
training for a position can be 
s tructured In a sequence lead ing 
to job qualification* 

2. Provide a means to ensure that 
trainees are initially assigned 
instructional materials which 
provide the taste knowledge pre- 
requisites before task practical 
training beqlns* 

3« Provide a means to ensure that 
when trainees do not mieet task 
knowledge prerequisites, they are 
provldsd with comMnts which In- 
clude such Information as which 
learning objectives hsvs not been 
attained, where the spsclflc 
informs tion rslated to those 
objectivsa can be founds and the 
idsntif ication of a test to be 
taken after restudy* 

4 . Provide a mieans to ensure that 
trainees sre assigned associsted 
practical training on a task 
after meeting taste knowlsdge 
prerequisites* 

5. Provide a means to snsure that a 
capability for matting multiple 
practical asslgnmsnts sxists in 
those Instances where task know^ 
ledge prerequisites have been met 
but task practical trsinlng can- 
not follow Imaedlstely* 
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TABLE 4, RBgUlRBHBNTS FOR AN EFFECTIVE OJT SySTBK (ContM.) 



C, inetructional Technology (Cont*d.) 

6. FrovLde a inean« to produce a ILat 
of study references and other re- 
dourcea required for each aaaL9n- 
nent r in formation as to what 
parts of the references are ap- 
plicable to the objectives of thtt 
assi9nBMntr a reference to the 
test which will be administered 
upon completion of atudyr and an 
et^timate of the time (contact 
hours or days) required for com^ 
plet^lon. 

7. provide a means to 9enQrate know- 
led9e tests randomly from selec- 
ted sets of parallel forms r and 
to format tests usin9 items ran- 
domXy selected from banks of test 
items indexed to learning objec- 
tives as determined in the task 
profic&ency training development 
process . 

8. provide for the construction r 
administration r scoring r evalua- 
tion r and performance data re- 
cording of OJT testing. 

9- Frovide a means of assisting 
supervisors in sequencing of job 
tasks for training on an indivi- 
dual basis r considering task 
difficulty, training opportunityr 
frequency of performance r produc- 
tion workloadr resources availa- 
bility r Prerequis i tes acqul red r 
and tralnin9 difficulty. 

10* Provide a means of allowing devi- 
ations from optiDun sequencing of 
task practical training assign- 
ments as production/operational 
workloads dictater and of record- 
ing these deviations in trainee 
records and training validation 
records. 

11. Provide a means of making assign- 
ments to modules of instructional 
mater ials r ind ividusl ited or 
trainer/instructor-delivered r for 
a task in a normal sequence which 
is determined during the couree 
development process to be opti- 
mumr and which cannot be changed 
by other than course developers. 

13. Provide an assignment strategy by 
means of which a traineer if not 
fully position qualifiedr has at 
least one Incomplete assignment 
which is related to a task iden- 
tified for his/her position. 

13. Provide data to support the deci- 
sions related to media selection 
which result from the application 
Of i$D principles to OJT develop- 
ment. 



14. Frovide data to support the deci- 
sions related to the sequencing 
of instructional activities which 
result from the application of 
I&D principles to OJT develop- 
ment. 

15. Frovide data to support the deci- 
sions related to the identifica- 
tion of training resource re - 
qu iremen ts wh ich resul t from the 
application of ISD principles to 
OJT development. 

16. Frovide data to support the deci- 
sions related to the development 
of r and revisions tOr instruc- 
tional materials which resui t 
from the application of ISD prin- 
ciples to OJT development. 

17. Frovide data to support the deci- 
sions related to instructional 
material val idatlon which result 
from the application of ISD prin- 
ciples to OJT development. 

18. Frovide data to All management 
levels within the OJT system 
vhich depict the results of the 
application of ISD principles to 
the OJT system. 

19* Frovide a means to identify the 
total set of learning objectives 
related to each task to be 
trained. 

30. Provide a means to establish and 
maintain the detailed training 
task requirements and associated 
minimum proficiency standards for 
each duty posi tion in a work 
center/section. 

31. Provide the capability to inter- 
face with computer-based simula- 
tion technol09y in the specif ic 
areas of establishing the re- 
quirements for simulation and in 
the collection and evaluation of 
both trainee and simulator per- 
formance data. 

D. ISD/Authoring Aids 

i. Frovide a means to identify each 
training taakr learning objec^ 
tiver and test item that might 
require modification when changes 
occur in weapon systemB, operat- 
ing procedures r maintenance pro- 
ceduresr etc. 

3. Provide a means of caciiltating 
the updating of instructional 
materials, tests, and task train- 
ing data utilized in the training 
system when required by changes 
to weapons sytems, operating and 
maintenance proceduresr etc. 
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TABLE 4, KBQUJFBKBNTS fOJt AN KPPBCTIVK OJT SYSTUt (C0nt*4«) 



0. IHfyAuthortng Aldi (Cont'd.) 

3. Provide ft mftftno of fftcllltfttlng 
the dldtrlbufclcn of »vlfllonfl to 
Inatructionftl mftterlalfl t testft^ 
and tftfik training d«ta utilized 
within the trdinlng eyflteM wh«n 
required by changes to iiieapena 
ays tend t operating and Mainte- 
nance procttdur«9f etc. 

B* Job Reading Training 

1, Provide a neane for diagnosis of 
trainee deficiencies in job read- 
ing skillet 

2, provide A tteane to Integrate with 
task proficiency training ^ the 
administration of job-relat«d 
reading skill training. 



IKSTltUCTIOKAL NAUAGENEHT 
P. Airman Training_Record 

1. provide a neans to establish and 
Maintain In a conon fonut^ an 
individual airman training record 
which will Include stai>dardized 
inEornatlon related to formal 
resident technical schools traln^ 
ing t task proficiency trelning^ 
anc illary training t war skills 
training f cross-utilization 
training , enroute training , Mid 
other base-level training # tthlch 
can be forwarded to each bAse of 
assignment throughout the air- 
man's career* 

2. provide a means to record the 
^attainment of job task skills 

within an Air Force Specialty 
when those skills are certified 
at the resident technical ecbool 
or FTDt 

3t Provide a measure of current 
position quallf lea t ion In each 
airman train ing record which 
reflects the task proficiency 
qualifications of tbe current 
position* 

4* provide a measure of current APS 
development In each airman train- 
ing record which reflects the 
numbers and types of positions 
held and degree of qualification 
attained in each* 



9* provide a mesne of Informing 
trslnsss of their progress toward 
proflcency on all tasks for which 
training Is being accomplished 
and which must be certified* 
Such Information should Include # 
as a minimum^ which milestones 
(knowledge and skill testst bsve 
been suoassfully met^ which have 
not, and the time which bas 
elapsed since each outstanding 
assignment was msda* 

6* provide a means to collect and 
maintain Information related to 
the personal attributes of an 
airman {Armed Services vocational 
Aptitude Battery (AHVAB) scores, 
sge, education^ pay grades renk^ 
years servlcs^ etc*) thst may be 
used In assessing tlis progress of 
each trainee toward proficiency* 

7* provide a means of Identifying 
those trainees Mho demonstrate 
In itiatlve In their progreas 
toward proficiency In those tesks 
Identified for their positions # 
so thet equlteble incentive 
awards cen be msde* 

e* provide a mesne of Identifying 
those trelness who do not make 
adequate progress toward profit 
Clancy In those tasks Identlflsd 
for tbelr positions, so that 
counsel ing and other remedial 
meaeures cen be provided* 

G* Diagnosis of individual Treinlng 
Requirements 

1* provide tools for evaluation of a 
newly aaslgned elrman*s pravlous- 
ly attained akllls against the 
teske to be performed in the 
position and produce a list of 
tasks for which training le to be 
provided, sequenced for manage- 
mant of the airman's progress 
toward job qualification* 

2* Generate lists of individual 
training requirements vhen crose- 
utllizatlon training f unit train- 
ing '^uch as "Prime Beef^" iter 
skills training^ small arms 
training^ ancillary training^ or 
othar requirsments occur tbat are 
not directly eesocieted with the 
eseignod position within the work 
csn te r/sectlon * 



M* Trainee Wensgement 

1* provide tools for making routine^ 
preferred esslgnmsnts of job 
taaka for Individuals bssod on 
optimum training aequence for 
full job quelif Icetion* 
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TABLE 4. REQUineHeilTS FOR AN BFPBCTlVe OJT ST8TBH tCont'dO 



Trainee Kanag^mi^nt ( Cont ' d • ) 

2« Provide LnCormatlon Cor th« inme- 
didtit supervisor to be used in 
makLn9 aBflL9runent6 of job tasks 
otit oC nornsl sequence when work* 
load condLtLons dictate and pre* 
requLeLtea are aatLsCLed. 

3« Provide assurance that routine 
assignnentB to learning activi' 
ties are made in a syateniatic 
nanner onc« a job task ia 
aasiqned* 

4« Provide assurance that task know- 
ledge prerequisites have be«n met 
before practical trainin9 on that 
task begins* 

S* Provide assurance that^ in the 
event that task practical train- 
in9 ia delayed # task knowled9e 
has not decayed to a leaa-than* 
acceptable level* 

6« Provide a rqeana to account Cor 
practical trainin9 tlMie accumula- 
ted Cor each task that occurs in 
the normal sequence oC learn in9 
activities COr that task« 

7« Provide Cor the collection # re* 
cord inqt and report in9 oC all 
trainee per£oraance data related 
to the learning activities which 
result in task proficiency* 

B* Provide information Cor the 
trainer related to the tra inee * a 
perCormance on all learning acti- 
vities associated with that job 
task prior to the start oC prac- 
tical trainln9« 

9* Provide for the recor<lin9 ot the 
date on which the trainer deter- 
mines that the trainee has 
reached an acceptable level of 
proficiency on job task perCor- 
mance « 

10* Provide to the innediate supervi* 
sor ior other designated certiCy- 
ing oCCicial) prior to the task 
certiCicatlon process # informa- 
tion related to the trainee* s 
perCornance on learning activi- 
t ies associated with that job 
task* 

11 « Provide support Cor the cert id- 
eation process throu9h evaluation 
of oral # written , and/or task 
proficiency testing In accordance 
with job proCiciency objectives # 
the record in9 oC results , dla9- 
nosis oC def iciencies^ and 
assignment oC appropriate reme- 
dial l«arnin9 activities* 



12* Provide support for the external 
evaluation process after profi- 
ciency on a task has been satis- 
Ciedf in the speciCic areas oC 
testin9 1 record ing # report in9 1 
and diagnosis of deficiencies* 

Unit Training Records 

1. Provide a capability Cor recor- 
ding training^ certiCication^ and 
qualiCication data At the work 
center /sect ion levels 

2* Provide siunmary reports of train- 
ing # cert iCicat ion # and qualifi- 
cation at the work center/sec- 
tion # squadron/ unit , base t and 
HAJCOK levels. 



TRAINIKG RedOURCES HAHAGSMEHT 

Training Resource Allocation 

1« Provide a ateans to establish and 
maintain an inventory oC training 
resources required to provide job 
qualiCication Cor every position 
on a base and required to conduct 
other base-level tralnin9« The 
inventory will include instruc- 
tional materials and technical 
data relevant to each task to be 
tralnsd t train in9 aida^ training 
devices f train in9 facilities^ 
operational equi|«ent# supplies^ 
and human resources required Cor 
traininq. 

2* Provide a means oC identifying # 
account ing Cor the utilisation 
ott and allocating scarce train*' 
Ing resources « Factors which 
will be considsred in the desig- 
nation oC a training resource as 
"scarce* include quantity^ criti- 
cality to mission accomplishments 
cost^ production backlog t and 
training backlog* 

3« Provide a means to determine 
training resource svailability so 
that a comparison can be made 
with individual and group train- 
ing priorities to enaure eCfi- 
cient scheduling* 

A* Provide a means oC holding in 
suspense records of train ing , 
certification^ and external eval- 
uation events which are deCerred 
because of non-availability of 
training raaourcee« 

5« Provide a means of race iving 
InCorotation CroAt trainers and 
instructors related to utiliza- 
tion oC training resources; e«g«s 
time use begins and ends^ spe- 
ciCic eventa^ etc* 
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TABLE 4. RtOUlRIHlHTB rOK AM IPPICTIVB OJT SYBTBH (Cont*^.> 



J. Tgaininq Reaourcft Allocation {Cont'd^) 

6« Provide A MAna ot AoalstLng th« 
aupervlBor Ln obtAlnLng training 
resource A for tAAk pro£ Ic lency 
trAlnlng which le norMlly Ache- 
duled by thA supervisor 

7* Provide A «AAn9 or supplying pro-^ 
jections qZ resource AvAllAblllty 
to thA schAdullng function f and 
of verifying avmllAblllty prior 
to AChedule pronulgatlon. 

e« Provide support for the osrtlf 1- 
cat Lon and external svsluatlon 
processes through Allocation of 
resources as regulredi^ 

9« Provide A meAns of Assisting 
supervisors In sequenc Ing task 
pro f Iclency practical train Ing 
for Individuals by providing cur-^ 
rent data on resources avalla* 
blllty. 

K. Scheduling 

1« Provl^ a means of forecast 1^9 # 
prlorltlsLitg^ and scheduling all 
train Ing f certification and rs-^ 
certification events on a base* 

2* Provide In formation related to 
the scheduling or training to the 
train Ing resource allocation 
funct Lon for the optimal allo- 
cation of such rssources* 

J. Provide f for all a^adrons/unlts 
on a basSf forecasts and sched* 
ules of training and recertifies- 
tlon events. 

4. Provide a capability for adjust- 
ment of training schedules by 
supervisory and managestent per- 
sonnel If required to conpensate 
for unexpected worlcloadSf contln- 
genclesf etc 

S+ Provide reports fr^r training sys-^ 
ten manageiBent which are needed 
to assess the effectiveness of 
the scheduling function.. 

6 * Provide A means for achedul Ing 
external evaluation events so as 
to mlnlotlze the Impact on work 
center/section production. 

7. provide Information concerning 
scheduling efficiency ror use In 
OJT cost accounting and control* 

Bi. Provide a means of Identifying 
alternatives to projected tramm- 
ing events for which resources 
will not be available. 



TitAiHino Btenn mwAfiBiiiMT 

Li. Training Quslltj Control 

1. iwvlde e single tralnln9 «snA9e- 
ment focal point at sach Air 
Force base wboee sole responslbl* 
llty Is to ensure thet all base 
personnel meet the tsAk prof 1-^ 
Clancy standards rsqulred to 
satisfy mission re<|utrements. 

2* Provide management controls nec-^ 
essary to ensure that OJT devel-^ 
opment Is conducted In compliance 
with Air Porcs ISD policy. 

3. Provide for adjuatttents In Job* 
site trslnlng progrAms to accom-^ 
modate changes In training rs* 
qvlrsments. 

4* Provlds a mssns of estAbllshlng 
psr f ormance stsndarda for unit 
OJT managarsf trslnerA, and other 
personnel directly Involved with 
on*ths-^)ob tretntng- 

5* Provlds systsmatlo task prof 1- 
clency evaluation procedures 
which are external to the work 
center/section In which teak 
proficiency Is certified* 

6. estsbllsh and Mlntaln baseline 
data for each work center/section 
to 6e used In the aesesement of 
unit training effectiveness* 

7* Provide analyses of the data 
generated by the training system 
to determine the effectiveness ot 
sll trslnlng conducted et the 
base level* 

e. Provide for sn exchsnge of Infer* 
metlon with aircraft and missile 
maintenance quality con trol f 
standardisation evaluation « and 
other quality assurance otganl- 
satlons f^t the purpoas of Iden- 
tifying training-related deflcl-^ 
encles In the operational envl* 
ronment.. 

9* Provide* for KAJCOH training 
managerSf analyses of the effec* 
tl veness of unit training In 
meeting mission requirements. 

10. Provide skanagement controls for 
the eddltlonf deletion, and revi- 
sion of tasks on the Master task 
list as required by chsnges In 
weapons systemSf contingencies # 
significant vsr let ions In work- 
loads reorgsnlsAtlonSf etc+ 
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TABLE 4. ttEOUlREMeifTS FOR AN BPPeCTlVe OJT SYSTEM (Cont*dO 



Training Qunllty Control (Cont'd.) 

11^ Provide manageMent confcrola for 
the addlfclon^ d«lefclon« and revl- 
t Ion o £ ta Bka on the poB 1 1 Ion 
train Ing requ lre«enfca 1 Iflfc re^ 
qu Ired by changes In weapons 
syafcena« confclngencita« algnlf i- 
canfc variations In workload t 
r€Organi2afclona« etc* 

12, Provide a neana of Identifying 
fchos« Individuals who possess fche 
afctrlbufcea of good fcralners* 

13^ Provide a means of monitoring 
fcralner quall£ lea ions to fcraln 
specific fcasko, 

14 ^ Provide Information related to 
tra i ner l^r formance in tra in ing 
specific fcasks^ 

IS^ Provide a Means of identifying 
and Measuring fcralner proficiency 
wlfch respecfc to fchose Instruc- 
tional procedurss and fcasks which 
prooote trainee proficiency^ 

16, Provide a means of establishing 
an audit fcrail which will allow 
correction of skill deficiencies 
in fcrainers it fcrainee perfor- 
nance so warrants^ 

«^ Training Cost Control 

1. Provide a cost account ing and 
control methodology which can 
utilise in formation from the 
tra in ing resource allocation t 
scheduling t fcralnee management* 
and ofcher sysfcsn functions to 
provide perioaic reports of costs 
m such training categories as 
OJT t anc lllary training • var 
skills training* and cross-'utill-' 
nation training, 

2 4 Provide a means of producing 
training resource availability « 
ufclli2ation« and cost data to OJT 
managers and others concernsd 
with fche analysis of fche effi- 
ciency ot training^ 

3 4 Provide cost accounting and re-* 
porting in fc^rms of trainee cosfcs 
and fcofcal cost* by squadron/unlfc 
and base* and by Air Porce Spe- 
cialty across Major Commands* 

4, Provide dafca for the development 
Of Air porce Specialty cosfc indi- 
ces which reflect fche efficiency 
of unifc draining programs^ 

5 4 Provide a capabil Ifcy to make 
training cosfc comparisons between 
similar squadrons/ unifcs* 



6* Provide analyses of the cosfc dafca 
generated by the sysfcem to deter- 
mine fche efficiency of all train- 
ing conducted afc fche base levels 

7 4 Prov Ide in £orma 1 on for he j us - 
t if Icafcion £or procurement of 
fcraln ing resources requ ired to 
accomplish draining afc fche base 
levels 

6, Provide cost analyses for use in 
forecasting resource requirements 
in support of budgetary decision 
making and projections of train- 
ing requirements^ 

9^ Provic ■! analyses of information 
related to training resource 
availability* task training dif- 
ficulty* task proficiency and the 
quantity and quality of training 
£or use in determining OJT capa- 
city for specific Air Porce Spe-* 
cialties filling various posi- 
tions in operational units and 
squad rons4 

IO4 Provide analyses of Information 
for HAJCOH and Air Staff training 
managers related to train Ing 
resource availability and utili- 
iation« and the specific numbers 
o£ airmen qualified to train 
specific tssks* for use in com- 
puting OJT capability in units 
and squadrons* 

11, Provide analyses of the ef f 1- 
ciency of the scheduling process 
considering forecasts of training 
to be accomplished during a peri- 
od, training accomplished* fre- 
quency of revision* etc* 

12, Provide a means of Identifying 
commonalities in tssks and/or 
positions in order to establish a 
job rotation policy that would 
provide cost benefits and other 
efficiencies to the Air Porce 
with minimum Impact on mission 
capability, 

13, Provide management for an incen-* 
tive awards program which recog- 
nises the benefit to the Air 
POrce of achieving training cost 
ef f ic iencies^ 

14, Provide the capability to simu- 
late or modsl the Air Force OJT 
system for the purpose of pro- 
jecting the costs associated with 
making training pipeline modifi- 
cations « Chang Ing missions* 
changing training loads « and/or 
revising other training t person- 
nel« and classification policies. 



40 



ERIC 



45 



TABLK 4, HtQUinWtHTB ton AM CrrsCTlVe OJT syBftM (cont'd.) 



Manag^w^nt inforiiation Interfaces 

I* Provide data concerning training 
requir<tftent& conpl«ted , certifi- 
cation t decertif ication^ and 
recer ti f i cation to th« Kainte^ 
nance Personnel subttysteAi of the 
KKICS data Byat«m« 

2* Provide for an exchange of infor- 
Ntation with aircraft and MisBile 
maintenance qual Lty control t and 
other production quality control 
or9anl2ation0# for the purpose of 
identifying procedural deficient 
c ies in the epeci f i CAt Ions for 
task performance. 

3* Provide Information related to an 
Airman * s current poai t ion qual i- 
£ i ca t i on f and his/ her curren t 
APS development to the Air Force 
Personnel 1>ata System as required 
in support of promotion policy, 
assignment policy^ classification 
policy, and other career develop- 
ment policies. 



4« ^ovlde • msAne to producs datA 
related to those tasks being 
trs ined on the job by vAr ious 
categories tsuch ss AF spocislty^ 
position^ pay grsde, vespon sys- 
tem# task commonsllty moross hTBt 
orgsnlsstlonml components etc* ) 
for occupstionsl snslysls. 

5 « Provide a means to produce da ta 
that represent the trslnlng 
sccoa|>l Ished within the Ojt sys- 
tem for sn Air Fbrce Spoclslty In 
order that the totsl training 
rmquLrements of the epdclslty My 
be snslysed for the development 
of the ST8. 
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III. ITS DESIGN ALTERffATIVBS/FBASIBILm ANALYSIS 



DeBcrlptlon of the Alternatives 

The requirements for an effective OJT system as defined in Table 4 
provided the basis for configuring three different ITS design 
slternativea for consideration in the Trade Studies Analysis during 
Phase II of this effort. The ITS functional model that resulted from 
the requirements definition process is shown in Figure 3. 

Five major processes that must be accomplished in order to conduct 
job-site training are sumiaarized in Figure 4. While each system 
requirement defined in Table 4 is applicable to one or more of these 
five processes^ many of the requirements are not esaential to meet the 
'^minimum'* requirements of an effective OJT syatem. However^ they are 
needed for the syatem to attain ma^cimum benefits. These minimum 
requirements have been grouped in Table 5 by their relationship to the 
five processes identified in Figure 4* Given enough manpower and 
sufficient management controla^ an effective OJT system could be 
designed to satisfy these requirements without any computer support or 
automation^ and the resulting system would offer considerable improve^ 
ment over the current Air Force OJT system. On the other hand^ many of 
the OJT deficiencies cited earlier could not be reaolved by addressing 
only these minimum requirementa. 

Data requirements are implicit in the need to provide better 
methods for identifying and updating specific training requirementa^ to 
provide more frequent and valid proficiency evaluation^ and to provide 
better methods for determining OJT cost and capacity. There is a 
fundamental requirement to standardize the data elements needed for 
solving such problems and to collect timely data for analyaia and 
program evaluation purposes. These two major OJT requirements^ data 
standardization and timely data collection^ indicated that data 
automation should be considered for ongoing support of the ITS system 
and should become a part of the design of the three alternative syatem 
configurations for the ITS trade studies. 

Another major consideration in the design of a more effective OJT 
system was that responsibility for all aspects of job-site training 
should reside within a management structure organized to ensure that the 
system requirements are met in a timely and efficient manner. Depending 
on the level of computer support^ responsibility for accomplishing the 
minimum requirementa identified would most probably reside within 
different functional areas on the base. In describing alternative 
approaches to configuring the ITS^ it is important to keep in mind thst 
Air Force functional responsibility for carrying out these minimum (and 
related) requirements shifts across functional areas with each 
alternative^ depending on the functions automated in that alternative. 
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iKmiMTtD TRAIKIMC SySTO* < ITS ) FOB AIR ppftcg OJT 



tj^^inIn:; PKVE LorHi:KT /del I VER Y 

TasK-by-Poftt tion Dfttft Base 

AKf m.^KtP^r tastk lint 

WorK c**!! t-** r poft i ti (in trai rii tig 

Job rotation ob;|f*rtives 
OrcuPA tion > I AhAlysm 

int**rf are 
STS irt^rfdre 

TftsK Proficipnry Objectives 

Ce r 1 1 f 1 r 1 1 on 

Inat t lonft 1 Technology 

Sequpncinq trftinina 

TasK Knowl<^1<ti» pr^r**tTuini tes 

Trainee PrpscriPtions 

OJT t#Etin<? 

Hflilia selection support 
Instructional i1e<:iqn support 
vallilation Sui*port 
Sli^ulAt ion interfaces 
AutbOrin<T al(i$ 

Job heading Training? 

Diaqnosis of df*ficiency 
Integration with task 
proficiency training 



airman Training Record 

Skill certification 
Positifvn Qualification 
Career developot^nt 
Proficiency pro^r^ss 
Prediction ^ata 

Diagnosis of Training 
R^qui regenta 

Training guid« 
Secondary positions 

Trainee Harflqefnent 

Progress tracking 
Performance data collection 
support for trainers 
Support for certification 
Support for evaluation 

Unit Training Record s 

Work center training and 
qualification records 
mainte nance 

Summary training records for 
squadron r base^ H^JCOH 



traihihc resources kanacehemt 

Training Resources AUocatton 

Training resourtres inventory 
Scarce resources accounting 
Deferred training record 
Resource utilization data 
Aesoutce availability 

projections 
Resource allocation 

scheduling 

Forecasting' prioritising! arid 

5ch*»duling training 
Scheduling effectiveness 

reports 
QC event scheduling 
Scht^duling efficiency analysis 
Resource availability vs 

training alternatives 



TRAIHINC SYSTEM KAHftGDIEtTT 

Training ftiality Control 

Training development controls 
Standards for OJT managersf 

trainerSf and others 
Systeoiatic task proficiency 

evaluation 
IHit training effectiveness 

analysis for KAJCOK 
Interfaces with production 

for training deficiency ID 
Training Management controls 
Trainer qualification 

management 
Trainer perforioance data 

Training Cost Control 

Categorical c09t accounting 
Training efficiency analysis 
AFS cost Indices 
Unit training cost conparlaon^ 
Training resource procurement 
Resource requirements 

forecasts 
Training capacity analysis 
Training cap4iblllty Analysis 

rotation cost benefits 
Incentive awards management 
Cost models for OJT 

Management Information 
Interfaces 



Training^ certification data 

for KHICS maintenance 

personnel subsystem 
Qualification and APS 

development data for ti>S 
Task performance procedural 

deficiencies for production 



Figure 3. Major Functional Areas and Components of the Integrated Training System 
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Figure 4. Five Processes Required for Effective Job-Site Training 
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TABLE 5. HINIMUM REQUIREMENTS FOR AN EFFECTIVE OJT SYSTEM 

Process Minimum Requirements 

Function from Table 4 



Identify Job Tasks for Position 

Develop Training for the Job Task 
Develop learning objectives 

and tests 
Sequence learning activi* 

ties 

Develop instruction 
Determine resource require- 
ments 

Diagnose Individual Training 
Requirements 

Deliver Training to the 
Individual 

Sequence/assign job tasks 
Assign learning activities 
Evaluate trainee skill/ 
knowledge achievement 
Provide feedback 
Track trainee progress 
Evaluate task proficiency 

Evaluate Training for the 
Position 



A3, Bl, Lll 

C8, C19, C20, Dl 

CI, C2, C4, Cll 

C3, C6, C8 
C6 



A8, C9, Gl, G2 



HI, H2 

CI, C2, C5, C12, H3, Hll, H12 
H4, H5, H8, HIO, Hll, H12 

C3, Hll, H12 
F5, F8, H9 
B2, F3, F4 

J8, K6, LI, L4, L5, U3, U4, 
U5, L16 



Alternative Design Considerations 

Three factors were held constant across &11 three alternative ITS 
design approaches: 

o Each alternative had to satisfy all of the minimum requirements 
for an effective OJT system* 

o Each job task identified as a training requirement in the 

system would be defined at the same level of specificity with 
the same performance standard(s) in each alternative* 

o The basic procedure for task proficiency evaluation would be 
the same in each alternative and would vary only in the types 
and level of support provided through data automation. 
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Several other factors were varied in developing the three 
alternative ITS design approaches; 

o The approach to atandardizing job task deacriptions and/or 
training requirements for each Air Force specialty 

o The extent of development ot job taak instructional materials* 

o The type of instructional management support to be provided to 
the trainee /trainer/ supervisor « 

o The level at which trainee performance data would be collected. 

o The capability to manage the training resources required in 
support of job taak training « 

o The amount of data and level of detail at which data would be 
collected for assessing system performance » 

o The extent of data automation to be provided. 

General Description 

During the initial phase of this effort, information was gathered 
which led to the definition of system requirements for the ITS and to 
the conceptual design of three alternative ways in which the ITS could 
provide data automation support for Air Force OJT» During the second 
phase of the effort » the alternative ITS design options were further 
refined to permit analysis of costs and benefits of each design 
alternative. All three of the ITS alternatives were designed to meet 
the minimum requirements for system ef fectivenesa^ which were oriented 
toward providing a large measure of responsiveness to mission 
requirements. The differences between the alternatives were mainly in 
the levels of support which the ITS could provide to relieve training 
managers and others of the burdens of the OJT administrative workload^ 
and in the type of capability the ITS would include to provide accurate, 
timely measurement and assessment of system effectiveness. 

The following system configuration alternatives resulted from 
consideration of the above factors! 

Alternative 1 . This approach would maximize the use of existing 
resources to establish the system and begin operations. Data automation 
would be restricted to those minimum requirements absolutely essential 
for an effective OJT system* Data collected related to training events 
and quality control evaluation would be limited and would be entered 
into the system at one of five designated collection points on a typical 
base* The potential for evaluation of overall OJT system effectiveness 
and efficiency would be minimal^ requiring extensive manual effort for 
producing suimnary reports. Cost effectiveness data related to 
resources, scheduling and other critical components would not be 
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available in an automated form* Scheduling and resource allocation 
would be done manually* Alternative 1 characteristics are summarized as 
follows ; 

o The use of existing instructional material resources would be 
maximized* 

o Data automation would be restricted to minimum essential 
requirements * 

o Limited job site training data would be collected at five 
locations per typical base* 

o Manual effort would be required to assess system effectiveness* 

o Manual scheduling and resource allocation functions would be 
required * 

Alternative 2 * This approach would incorporate modifications and 
improvements to the current OJT system that must occur in order to 
realize significant gains in satisfying the broader requirements for an 
effective OJT system* Under this approach, such improvements would be 
addressed by placing more emphasis on developing and redesigning 
training materials tailored to the specific requirements of training the 
task on the Job and by providing a better capability to evaluate the 
effectiveness and efficiency of the OJT system* This latter improvement 
would require that appropriate data be collected during each training 
event and also during the quality control evaluation process to be 
utilized in training management » program evaluation, and program assess- 
ment* Data automation would be provided to support instructional 
management and data collection for these functions at one of 20 
designated collection points on a typical base, located as close as 
practicable to the major concentrations of training being accomplished 
in the work areas* Some data automatioi support would also be provided 
for scheduling and allocation of training resources* Several standard 
computer reports would be produced concerning OJT effectiveness and 
efficiency* Alternative 2 characteristics are summarized as follows; 

o There would be a greater quantity of new and redesigned Job* 
site instructional materials* 

o Data automation would be increased for most functions* 

o Support for instructional management and data collection would 
be increased to 20 locations per typical base* 

o There would be some standard computer reports of OJT 
effectiveness and efficiency* 

Alternative 3 * This approach would attempt to satisfy as many of 
the defined requirements for an effective OJT system as possible by 
utilizing the full capabilities of computer technology where feasible* 
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Research would be performed to develop new computer-based oethodolog^ieB 
to use existing occupational survey and other task data for defining Air 
Force upecUlty trfllnlnR rcqntrraentn for OJT And to devplnp new task 
iUidlyLlc t ttchnltiuttn fur coiii|>ut<sr-*iiuppDrted generation of position 
training requirements, OJT development priorities, and AFS naster task 
lists. Extensive effort would be devoted to developing training 
materials for a majority of the job tasks identified as position 
training requirements. Multi-media and/or computer-abased instructional 
materials would be utilized where appropriate* Data would be collected 
on elements of each training event for instructional management and OJT 
program assessment purposes. Detailed performance and time data would 
be recorded for each training event as the training occurs, with a 
minimum of 75 designated data entry points on a typical bas«. A variety 
of computer reports would be produced on a regular basis and an ad hoc 
report generation capability would be provided for inquiries of ITS data 
bases for unique requirements. An automated resource allocation and 
dynamic scheduling capability would be incorporated for planning, 
management, and evaluation purposes. Alternative 3 characteristics are 
summarized as follows i 

o Full computer support of all functional requirements, including 
job-site training delivery, would be provided. 

o A large-scale training design and development effort would be 
involved • 

o Detailed training data would be collected when and where 
training occurs at up to 75 locations per typical base. 

o Enhanced OJT reporting and information retrieval capabilities 
would be included. 

o Automated scheduling and resources allocation would be 
incoporated for OJT management. 



ITS Trade Studies Analysis 

A series of trade studies were performed during the second phase of 
this effort. Information from these studies was used in a trade study 
of the three alternative system design configurations in order to 
recommend an ITS option that would best meet the needs of the Air Force 
in the support of on-the-job training. The emphasis in the trade 
studies was to develop estimates for comparison purposes across a 
variety of system alternatives^ rather than to analyze "final" decisions 
related to the ultimate syatem design or capabilities of the ITS. 
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To facilitate making reliable compariBonB of the total costs of the 
ITS alternative design options, a "typical base" had to be defined in 
terms of estimates of population, numbers of squadrons/units, numbers of 
work centers/ sections , tasks to be trained, and other estimated values 
which could then be extrapolated Air Force-wide. It should be 
recognized that the "typical" estimates used were, of necessity, derived 
values, and that they were made only for the purpose of comparison of 
the ITS alternatives themselves. 

The approach taken in estimating the total cost of each ITS 
alternative included the following steps: 

o Segmentation of base-level. Air Force specialty, and Air Force- 
wide costs into the five major OJT process functions, and the 
ITS program management and computer support functions^ 

o Calculation of the estimated costs of ITS-supported OJT on a 
typical base* 

o Extrapolation of the estimated base-level costs Air Force-wide^ 

o Determination of the discounted annual life cycle costs for 
each alternative ITS configuration. 

The projected costs for each ITS alternative were developed by 
first estimating the number of manhours required to support each of the 
requirements for an effective OJT system. Labor costs were calculated 
by multiplying the estimated manhours for a specified requirement by the 
standard composite rate for the pay grade as defined in AFR 173-13, USAF 
Cost and Planning Factors Regulation. Equipment costs were calculated 
by averaging data obtained from computer industry sources for similar 
equipment. Computer support costs for personnel, equipment, and 
software were also developed for each alternative. Appendix B provides 
a description of the process utilized to develop computer support costs, 
and the results of the computer support trade studies analysis. 

The detailed cost estimates for each alternative at a typical base 
were then categorized by major ITS processing function for both 
investment costs and recurring operational costs. Investment costs are 
those one-time costs which are incurred to develop and implement the 
system, such as purchase of computer equipment, development of computer 
programs, and establishing the master task list. Recurring operations 
costs are those annual costs which are incurred repetitively over the 
life cycle of the system, such as man-hours expended for OJT quality 
control and computer operations. ITS program management costs include 
all costs that could not be directly allocated to one of the OJT process 
functions or the computer support function, such as scheduling, 
effectiveness and efficiency analysis, unit training effectiveness 
assessment, and capability and capacity analysis. 

The base level cost estimates were extrapolated to Air Force-wide 
estimates and included with other coats calculated at the total Air 
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Force level. In addition^ eBtimated R&D costs for ITS Alternative 3 to 
develop a methodology to convert occupational survey data into OJT 
requirementB and to develop nev survey techniques vhich vill result in 
the automatic generation of position training requirements » OJT 
development priorities^ and AFS master task lists vere included. The 
estimated costs Air Forced-wide vere developed for each ITS processing 
function by categories of R&D costs ^ investment costs ^ and recurring 
operational costs. 

The total discounted costs for a projected life cycle of 20 years 
using a standard DOD discounting procedure vere estimated to be as 
f ollovs; 

o ITS Alternative 1 - $1^198.20M 
o ITS Alternative 2 - $ 777. 82M 
o ITS Alternative 3 - $ 670. 85M. 

Alternative 3 is seen to be the preferred alternative based on cost 
alone* This is primarily due to the trade-off betveen labor costs 
expended annually versus the cost of automation to replace functions 
accomplished manually. Alternatives 1 and 2 vere estimated to require 
significantly lover investment costs for automation vith greater use of 
Air Force personnel required annually throughout the life cycle of the 
system. On the other hand^ Alternative 3 would require a greater 
capital investment in the early years to automate functions^ thus 
reducing significantly the lab^r expended annually throughout the life 
cycle. 

A comparison was also made of the benefits to be derived from the 
ITS alternatives relative to the requirements defined in Table 4* Bach 
alternative's components vere analyzed vith respect to 14 areas of 
quantifiable output measures; 



o 


Accuracy 


o 


Production or 








productivity 


o 


Availability 










o 


Quality 


o 


Implementability 








Maintainability/ 


o 


Reliability 


o 






Controllability 


o 


Safety 


o 


Manageability 


o 


Security 


o 


Morale 


o 


Time savings 


o 


Operating efficiency 


o 


User acceptance. 



The analysis indicated that ITS Alternative 3 vould be capable of 
providing the greatest overall benefit to the Air Force through its life 
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cycle. The following beneficial interim products^ which would be 
avsilsble for use in the operational environment prior to full-scale 
implementation y were also identified; 



o Air Force specislty task lists for OJT. 

o New occupational analysis techniques • 

o OJT development tracking aids . 

o Enhanced structure in OJT processes . 

o Trainer performance assessment measures. 

o New airman training record . 

o Quality control procedures for OJT. 

o New base-level training management organization • 

o Unit training effectiveness measures. 

Also addressed in the trade studies were the technical ^ economic ^ 
and operational feasibility of all of the defined ITS alternatives. 
Since potential alternatives judged not technicslly feasible were 
discarded during the development of design options » the three ITS 
alternstives that were compared were considered to be technically 
feasible. In addition^ for the purposes of comparing costs across 
alternatives y all appeared in varying degrees to be economically 
feasible. Operstional feasibility was assessed in terms of the 
potential for success » and was judged to be high for all three design 
alternatives examined. Based on the results of the trade studies^ ITS 
Alternative 3 was selected for further system definition and 
specification during the finsl phase of this effort. Two additional 
analyses focusing on Alternative 3 were slso conducted and dsta provided 
to the Air Force. The first of these was a cost^benefit analysis to 
compare the present OJT program with the ITS Alternstive 3 
configuration. The second was an ITS development site alternatives 
anslysis to examine the suitability of various Air Force bases proposed 
by the Msjor Commsnds for ITS development and demonstration. 
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IV. DEVELOFMBMT OF ITS SYSTEM SFECIFICATIOHS 



General DlBCUBslon 

Ab Btated in Bectlon I» the primary purpoBe of thlB effort waB to 
conduct a Bystem definition Btudy for preparBtion of a ByBtem 
Bpecif ication that would enable the Air Force^ in a subBequent effort^ 
to develop and demonBtrate a computer-baBed Integrated Training Syatem. 
This ByBtem waB to addreBB requirements for the management, development^ 
and evaluation of Air Force on-the-job training. This involved an 
iterative proceBB illustrated in Figure 5 in which each phase of the 
project augmented or expanded the conceptual design of the ITS, leading 
to the development of a set of system specifications* Out of this pro- 
cess evolved key building blocks that became major controls for the 
ultimate design of the specifications* 

When this effort was initiated^ the Air Force provided an illustra^ 
tive model which described major OJT problem areas and Job^site training 
concepts that should be explored* During the initial phase^ previous 
research and development (R&D) efforts related to problems in OJT were 
analyzed and the results incorporated into the field survey instruments 
and the improved model for management of task proficiency training that 
evolved from the base visits* From these sources and controls » 
requirements for an effective OJT system were derived* 

In order to ensure that the system would be compatible with and 
meet the requirements of the Air Force and its OJT management 
organization, review and approval decision points for OJT functional 
managers were inserted at the completion of the requirements definition 
and the trade-off analysis phases » and during the preparation of the 
design specifications. 

The key building blocks that became major controls for the design 
of the specifications were as follows: 

o The general procedural model for management of task proficiency 
training (Figure B-6» Appendix B) that evolved from the initial 
illustrative models the HQ USAF» MAJCOH» Separate Operating 
Agency (SOA), and base visits » and the review of previous R&D 
for OJT became the source of many of the system functional 
requirements* This model also provided a basis for analyzing 
each ITS configuration alternative and defining ITS data 
requirements * 

o The requirements for an effective OJT system (Table 4) defined 
the functional controls for the system and its corresponding 
data requirements. 

o The minimum requirements (Table 5) established a major control 
for each alternative design option analyzed in the trade-off 
analyses. 
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Figure 5. Iterative Process for Development of ITS System Specifications 
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o The functional data base that resulted from organizing data 
requirements into major functions to be supported by ITS 
provided the basis for estimates used in the trade-off 
analyses* This^ coupled with the preferred ITS design 
alternative (Alternative 3)» provided essential controls for 
the description of the ITS functional characteristics. 

As a result of the analyses and design accomplished throughout the 
process described in Figure 5» it was determined that the ITS to support 
on'the-Job training should be designed to address the following 
objectives 1 

o Increase individual and unit performance to enable the Air 
Force to improve combat readiness* 

o Redirect the emphasis in Air Force on-the-job training away 
from general career development and toward Job qualification. 

o Utilize more efficient methods of identifying the training 

requirements for airmen to become fully qualified in their duty 
positions * 

o Implement new techniques for performing valid^ standardized^ 
and more frequent task proficiency evaluations^ and for 
assessing unit training effectiveness to ensure that training 
quality control is maintained* 

o Support new methodologies for determining OJT cost and unit 
capacity to conduct Job-site training. 

o Support new technology in the delivery aspects of Job-site 
training^ thereby increasing proficiency with no significant 
increase in training time* 

o Redirect the activities of training technicians (AFSC 751X2) 
away from current administrative duties » toward duties and 
responsibilities related to training evaluation and the 
application of a valid training methodology* 

o Simplify the administration of OJT* 



ITS System Description 

System Overview 

The Integrated Training System is a computer-based system that will 
provide support » at all levels of management » of Air Force OJT in the 
following functional areas (see Figure 6)* 

55 



59 




Figure 6. Functional Relationships within the Integrated Tr^iiaing System <ITS) 



o Job-Site Training Requirements; This functional area of ITS 
addresses defining the training required to translate the 
requirements of the mission into unit/squadron mission 
readiness » and includes Air Force specialty training 
requirements by Job> duty position » and task* 

o Development; This functional area pertaina to the production 
of all training materials that provide the knowledge and skills 
needed for full qualification in a duty position* 

o Delivery; Instructional delivery is the means by which the use 
of training materials can be controlled to result in a 
predictable outcome; i*e*» task proficiency* 

o Management: The management aspects of ITS address training 

resources utilization* trainee progress tracking* instructional 
management, duty position qualification, and management of the 
training development processes* 

o Evaluation; This functional area encompasses the assessment 
and control of the cost effectiveness of job-site trsining and 
the System which supports it* 

Each of these functional areas was examined in depth to provide an 
analysis of requirements for an effective Air Force OJT system (Table 
4)* The analysis also identified major training and training management 
processes to be performed and the general information requirements in 
support of these processes* The information requirements were organized 
in terms of an ITS functional data base that evolved during the project 
and corresponding groups of requirements that must be Supported for an 
effective OJT system (see Figure 7)* Each ITS functionsl data base was 
defined as follows; 

o Master Task List (MTL): Contains all job taaks for each 
specialty within the Air Force* It would be utilized to 
standardize the identification of all job tasks and provide a 
unique description of each task for universal uae throughout 
the Air Force* 

o Position Training Requirements (PTR)t Contains the 

identif icstion of each position in a work center/ section and 
each job task on the master task list that is applicable to 
that position* It would be used to identify the training 
requirements for each position and to provide a basis for 
generating an accurate diagnosis of the training required for 
an individual airman* 

o Master Instructional Management/Delivery Control (MIMDC): For 
each job task identified in the MTL» the MIHDC contains a 
detailed description of each atep in the training process 
required to meet task proficiency objectives* It also includes 
the controls essential for proper sequencing of the training 
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and £or the collection o£ historical data required to measure 
trainee progress and management* It would be the basis £or 
directing and controlling all o£ the learning activities ot 
each trainee* Once a task training requirement vas defined and 
entered in this data base« it vould be available £or use 
throughout the Air Force by all units providing training in 
that job task* 

o Airman Training Record (ATR): Contains a historical account o£ 
all training completed by each airman in the Air Force* 
Included vould be information related to resident technical 
schools trainings job task proficiency trainings ancillary 
trainings WARSKIL trainings cross-utilization trainings and 
other base-level training* This record would be forwarded to 
each base of assignment throughout an airman* s career and would 
be the basis for diagnosing individual training requirements 
for the airman and for providing various summaries of unit 
training performance and status* 

o Individual Training Requirements (ITR)t Contains a detailed 
account of the job task training required for an airman in 
his/her assigned duty position once the training needs 
diagnosis has been accomplished* It would be used to provide 
general training status and control trainee management 
functions while an airman is assigned to a duty position and 
would be closed out upon reassignment to another position or 
base * 

o Airman Instructional Management /Delivery Control (IMDC); When 
a Job task is assigned to an airman for trainings the task 
training requirements for that task would be located in the 
MIMDC and linked to a separate data file for that airman* It 
would be used to direct and control the learning activities of 
a specific trainee and to provide information concerning these 
activities to his/her trainer and/or immediate supervisor* 

o Training Resources (TR): Contains a detailed account of the 
training resources required to provide job qualification for 
every position on a base and those required to conduct other 
base-level training* It would be the basis for the system to 
schedule and allocate resources and to provide the data for 
analysis and monitoring of OJT costs^ capacity^ and capability* 

o Quality Control (QC); Contains a record for each job task and 
the identification of the individual airman for whom 
certification to perform the job task has been completed* It 
would be the basis for selecting job tasks for the external 
evaluation/quality control requirements of job^site training* 

o Unit Performance Summary (UPS); Contains a summary of the task 
certifications that have occurred and all training events 
completed within a given work center/section during specified 
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time intervals* This would provide the basis for analyzing 
training effectiveness in relation to mission priorities* 



o Master Task Training Requirements (MTTR); Contains a detailed 
description of each Job task for each AFS, the status and 
progress of development of all instructional materials and 
related documents, controls* and tools for building the MXMDC 
data base, and a catalog of all materials related to the 
training of a task« 

o Base Level Training Requirements (BLTR); Contains the 

identification of each ancillary training course and other 
training events not related to an AFS. It would be used to 
identify the non->AFS training requirements for an airman while 
assigned to an Air Force base. 

o Career Development Course Requirements (CDCR); Contains the 
Identification requirements and controls for each Career 
Development Course in the Air Force. It would be used to 
identify CDC requirements for an airman while assigned to an 
Air Force base. 

The identification of components of each major functional area of 
OJT and their support requirements were the result of analyzing OJT 
subfunctions, processes, and their associated information and data flow 
requirements. ITS subsystems were defined that represent the 
distribution of the components into manageable subsystems, described 
briefly as follows; 

o OJT Management Subsystem; Provides a means to identify the 

training required for full qualification in every duty position 
in the Air Force, a means to monitor and control the progress 
of individual trainees toward such qualification and other Air 
Force base-level training requirements, and to siiLplify OJT 
admin is t ra t i on . 

o OJT Development and Delivery Subsystem; Provides a means to 
produce the materials and media which will develop in trainees 
at the Job site the knowledge and akills required for full 
qualification in a duty position, and ^ means to deliver all 
instruction specified for certification on a Job task. 

o OJT Evaluation Subsystem; Provides a means to evaluate the 
efficiency and effectiveness of the Air Force OJT system, and 
ensure that training quality and cost control are maintained. 

o Computer Support Subsystem; Provides data automation support 
for all functional areas of the Air Force OJT system. 

o Personnel and Support Subsystem; Provides a means to identify 
and qualify personnel who will manage^ supervise, and operate 
the ITS and personnel who will develop and conduct OJT, as 
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well as the means to provide logistiCB and inaintenance support 
for Bystem operation. 

The relationships between Air Force mlsBion requirements » Job-site 
training! and the ITS are illustrated in Figure 8* 

ITS Functional Characteristics 

Each of the ITS subsystems was organised into functional components 
as illustrated in the ITS specification tree in Figure 9* Major 
functions that would be supported by the ITS are also illustrated. A 
general description of each component and the functional data bases 
required to support each are provided in the following sections. 

OJT Management Subsystett . The OJT management subsystem of the ITS 
would provide a means to (a) identify Air Force specialty (AFS)» duty 
position* ancillary and related base-level training requirements* 
(b) manage airman training progress toward task proficiency and full 
qualification in a duty position^ and (c) manage the allocation of base 
training resources and the scheduling of training* The components of 
the OJT management subsystem and their major functions are 

o Training Requirements Management Component 

AFS master task list (MTL) . 

AFS performance requirements* 

AFS training requirements. 

Position training requirements (PTR). 

Ancillary/additional duty training requirements. 

AFS career development course (CDC) requirements. 

Master task training requirements (MTTR) . 

o Airman Training Management Component 

Airman training record (ATRX 
Training control identification* 
Training needs diagnosis* 
Individual training requirements (ITR)* 

- Training progress management* 

o Training Resources Management Component 

Inventory establiahment • 

Training scheduling* 

Training resources allocation* 

- Resources availability monitoring* 

a. Training Requirements Management Component . The training 
requirements management component of the OJT management subsystem 
provides identification and specification of all performance and 
training requirements of the Air Force specialties identified for 
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incluBion in the ITS development and demonstration program* In 
addition^ this component provides identification of all ancillary, and 
other base*- level"^conduc ted training required by personnel assigned to 
the ITS development and demonstration base who hold any of the 
designated Air Force specialty codes* The data produced by the 
functions that comprise this component will also provide the necessary 
foundation data for diagnosing individual trsining requirements, monitoring 
individual training progress^ developing OJT instructional materials^ 
and evaluating individual task proficiency and training system 
effectiveness * 

The functional data bases incorporated to support the operation 
of this component of ITS are the Air Force master task list (MTL)» 
master task training requirements (^TR)^ generic and operational 
position training requirements (PTR) , base-level training requirements 
(BLTR)» and AFS CDC requirements (CDCR)* The ITS includes the 
capability to allocate resources for, schedule entry into, and monitor 
completion of training events associated with ancillary and other base- 
level training requirements, including CDCs * The training requirements 
in these areas are adequately identified and specified by current Air 
Force and MAJCOM/SOA policies* Task performance and training 
requirements related to assigned duty positions within Air Force 
specialties are, however, not adequately identified and specified by 
current policies* This has resulted in marked deficiencies in defining 
measures of OJT effectiveness^ obtaining task proficiency training and 
certification standardization, and managing individual progress toward 
duty position qualification* These deficiencies would be corrected 
through the development and demonstration of the methodologies and 
supporting program software necessary to create the functional data 
bases associated with this component of the ITS OJT management 
subsystem* 

o A master task list for each Air Force specialty identified for 
inclusion in the ITS development and demonstration program* 
The master task list should be derived insofar as possible from 
existing Air Force sources containing task identification and 
performance data* Each task identified on the master task list 
must be described at a level of specificity sufficient for 
individualized task proficiency training and performance 
certification* 

o Position training requirements (PTRs) , both generic and opera*- 
tional, for each duty position or job performed in work centers 
identified for inclusion in the ITS development and 
demonstration program* These are intended to be generated 
from the Air Force master task list when practicable* The 
generic PTR lists those tasks, drawn from the master task list, 
that represent generic duty positions within an Air Force 
specialty* The generic PTR must be capable of being 
transformed into an operational PTR that lists the tasks 
actually performed in the development site work centers* 
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Base-level training requirements (BLTR) for all personnel 
possessing the designated AFSs who are assigned to the 
development work centers* The base-level training requirements 
functional data base Identifies all Air Force and HAJCOM/SOA*- 
dlrected ancillary, and other base^level-^conducted^ non- 
speclalty'-related training requirements. These requlrementa 
will be Identified by a system of standardized Air Force course 
Identification numbers^ and the requirements description should 
Include at a minimum the course number^ tltle^ and renewal 
requirements data. The base^level training requirements list 
will group requirements based on common relationships such as» 
but not limited to» base and unit of assignment^ Air Force 
specialty^ pay grade^ and assigned duty position. 

Career development course requirements (CDCR) for all 
designated Air Force specialties authorized within the work 
centers. These requirements can be Identified and extracted 
from current Air Force Extension Course Institute 
documentation^ and must reflect current CDC requirements by 
skill level within the designated Air Force specialties. 

Master task training requirements (^f^TR) data base containing a 
detailed description of each task contained on the master task 
list (MTL) for each Air Force specialty Included In the ITS 
development and demonstration program. The MTXR should contain 
a complete description of each MTL task to provide a means of 
managing the status and progress of all training development 
activities related to the task and to the specialty. The MTTR 
should be designed to accept data In an Iterative fashion from 
all ITS functions Involved In the Identlf Icatlon^ 
specification, definition^ development^ and delivery of Air 
Force task-^speclf Ic training. The description of each task In 
the MTTR should Include at least a job task Identification 
code» the task tltle^ the weapon system/equlpment/f unctlonal 
area, prerequisite task(s) or sub^task(s)» performance 
specifications and proficiency requirements^ data source 
references, training resource and job literacy requirements^ 
performance/learning objectives^ and task knowledge and 
proficiency test Items. Task performance and proficiency data 
could be generated and collected In a format compatible with 
the unified data base (UDB) or other appropriate data bases. 
The MTTR would also Identify the agency having development 
responsibility and provide a means by which to establish, 
assign^ and monitor rhe status of production milestones related 
to training development activities. The MTXR can also provide 
a directory service Identifying all Air Force and other 
Department of Defense (DOD) agency users of ITS training 
development products. 
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b. Airman Training Manageittent Comgonent , The airman training 
management component encompasses the functions of 



o Airman training record (ATR). 

o Training control identification, 

o Training needs diagnosis . 

o Individual training requirements (ITR) * 

o Training progress management. 

These functions provide for (a) the identification of a duty posi- 
tion to which an individual airman has been assigned, (b) a diagnosis of 
the training required for that airman to become fully qualified in that 
position, and (c) the control of the job/task assignments necessary to 
ensure that each training requirement has been assigned, certified, and 
entered in the ATR. 

The airman training record function is intended to provide a 
complete record of all training received by an airman during his/her Air 
Force career. The training record must be maintained in a manner that 
will permit other system components and/or authorized Air Force 
personnel to determine accurately which job tasks an airman is certified 
to perform, as well as to diagnose the training needs for the airman 
with respect to a duty position. It is a permanent record of training 
that would "flow** with the airman from base to base throughout his/her 
career in the Air Force, Information that is required to be maintained 
is as follows: 

o Personal data from the airman PDS record: Social Security 
Account Number (SSAN), name, rank, date of birth, PDS record 
identifier, basic airman data, education/qualification data. 
Armed Services Vocational Aptitude Battery (ASVAB) scores. Job 
literacy score, job information data, duty status, AFSC data, 
etc, 

o Formal training data: the standardized course identification 
(ID) and date completed, etc, for each resident technical 
course or FID course successfully completed^ 

o Ancillary training data: the standardized ID and date last 
completed, etc. for each ancillary training event completed, 

o Generic position data: the generic position ID, dates held, 
and indication of full position qualification, etc, for each 
position held during the airman's career, 

o Job task data: the job task ID, current task certification 
status, date of last certification, organizational ID for the 
unit of last certification, and the unit where training was 
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accompliahed^ SSAN for the trainer of record^ and certifying 
official of record^ etc* for each Job task in vhich the airman 
ia or once vas certified to perform. 

o Trainer qualification datat the Job task ID^ date of trainer 
certification^ and organizational ID of unit where 
certification vas achieved^ etc. for each Job task on vhich the 
airman is certified to train others* 

o CDC data; the CDC number^ date assigned^ date completed^ etc. 

A major function of the ITS is to maintain and control the progress 
of each airman towards full position qualification. This cannot be 
accomplished without establishing and maintaining an accurate account of 
individual training requirements (ITR) for that individual and the 
status of the individual on each Job task recorded in the ITR. The ITR 
is envisioned as an interim record which augments the ATR to provide a 
complete training record of both the training in progress and training 
completed for the airman. 

The ITR for each airman is designed to maintain an accurate account 
of the certification status of each Job task recorded in the ITR. The 
following task training status conditions have been identified as 
minimum requirements to be maintained within this ITR function; 

o Task not trained * 

o Task previously certified at different base* 

o Task previously certified at present base* 

o Task knowledge training in process * 

o Practical task training in process * 

o Training completed but not certified* 

o Task certified in current position* 

o Task certification expired * 

o Task recertified in this position* 

o Task recommended for decertification* 

o Task decertified* 

o QC followup action in progress. 

Another major function of this component is to provide a means to 
manage all instructional activities and deliver all instruction required 
for certification on a Job task and to provide access to all system 
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functions In support of certification, recertlf Icatlon, and the training 
quality control activities of the ITS. Six levels of control were Iden-^ 
tlfled within the system to provide an effective management scheme for 
each Job task (see Figure 10). A general discussion of the relationship 
between each level of control within the system and Its primary purpose 
follows; 

o Individual training record control; The ITR Itself would be 
the highest level of control within the system and would 
represent a collective description of all Job tasks to be 
performed by the airman In an assigned duty position. The 
diagnosis function Identifies the Job tasks for which training 
Is required. The task training certification status would be 
maintained at this level of control for each Job task. 

o Job task control; A current account of the certification 

status of each Job task would be maintained within each ITR as 
the next level of control. As each Job task Is activated for 
training or for recertlf Icatlon/quallty control evaluation, 
control of that Job task would be transferred to the 
appropriate set of specifications as defined In the MXMDC. 

o Task training assignment control; This level of control would 
establish the IMDC file for an airman for the Job task 
activated. It would be used for (a) Identifying conimon sub- 
tasks that could be bypassed within this Job task that were 
completed successfully while In training for another Job task, 
(b) assigning IMD modules (training, certification, 
recertlf Icatlon, quality control evaluation) In the sequence 
specified, and (c) communicating changes In training status to 
the training progress management function. 

o Instructional management and delivery module control; An IMD 
module Is defined as a complete, self-'contalned set of 
Instructional activities , tests , procedures , resource 
requirements, controls, etc. related to a training module. It 
could represent a subtask or a complete task If no subtasks had 
been Identified. It could also relate to a task certification 
module, a recertlf Icatlon module, or a quality control 
evaluation module. IMD modules when combined would form the 
complete task training requirements for a Job task. This 
control function Is required for assigning Instructional 
activities In the sequence specified and for providing 
notification of IMD module completions to task training 
risslgnment control. 

o Instructional activity control; An Instructional activity Is 
defined as one of the essential steps In the task training 
process leading to task certification. These steps and the 
sequence of execution are (a) trainee meets criterion on 
knowledge prerequisites after completing study assignment, 
(b) trainer provides a demonstration of the task (optional), 
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(c) trainee practices the task, and (d) trainee performs task 
within specified proficiency standards* 

o Instructional segment control: An instructional segment is 
defined as the smallest unit of a Job task training process 
which contains all of the assignments necessary to achieve a 
particular training result. 

In order for the system to achieve these functions^ a detailed set 
of structured specifications related to all of the instructional 
activities required during the training process would need to be defined 
in a master instructional management and delivery control (MIMDC) data 
base* The controls described previously^ as established in the MIMDC, 
would become the primary means for the system to manage and control the 
instructional process from the time that a Job task is activated until 
certif ication/recertif ication is achieved by the airman or the quality 
control external evaluation function is completed* In addition, 
functions would be included to provide notification of training status 
to the training progress management function at appropriate checkpoints 
in the training process and to record all trainee performance data for 
use in the OJT evaluation subsystem. This would be accomplished by 
linking the task training requirements for a Job task as established in 
the MIMDC to an individual airman when the Job task is activated for 
training to provide a means to manage and control the Job task training 
process to completion. This can be accomplished with an airman 
instructional management and delivery control (IMDC) data base for each 
Job task consisting of the following three sections: 

o Specifications section: Provides the detailed instructions and 
specifications for training the Job task as extracted from the 
master instructional management and delivery control (MIMDC)* 

o Control section: Provides the control information needed to 

reflect an accurate accounting of task training status^ 

progress^ time in training for each control category^ 
outstanding assignments » etc. 

o Performance section: Provides a record of each instructional 
assignment, response^ disposition^ etc. and summary data for 
each control level achieved. 

c. Training Resources Management Component * The training 
resources management component was designed to provide a means to 
establish an inventory of training resources » to track the status of 
each resource^ to schedule resources^ and to generate usage data for 
training cost» capability, and capacity analysis. The following 
functions would be provided: 

o Establish^ an inventory of all training resources on a base^ 
including location^ responsible organization^ etc. 
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o Maintain an inventory of qualified trainers for each Job task 
trained on that base. Record data related to each trainer 
assignment for training and QC evaluation functions. 

o Maintain an inventory of scarce training resources that require 
allocation and tracking. Record data related to usage during 
allocations, non-availability, etc. 

o Process requests for allocation of reaources. 

o Schedule training resources, ancillary/additional duty training 
events, certification/recertification actions , and training 
quality control evaluations. 

o Allocate testing devicea and transfer control to the trainee 
evaluation component. 

o Allocate media and transfer control to the OJT delivery 
component . 

o Restore resources to ''available for use** status when notified 
that the instructional activity or training event Is completed. 

o Record changes in status due to non-availability conditions 
such as preventive maintenance, equipment up/down, etc. 

OJT Development/Delivery Subsystem . The OJT development and 
delivery subsystem was designed to produce and maintain instructional 
materials to support Job-^site training. It would also provide the 
mechanisms necessary for managing the development and delivery of 
training materials to support task proficiency training and 
certification. The two components envisioned for this subsystem and the 
functions that comprise them are described in the following paragraphs. 

a. Training Development Component ; The training development 
component of this subsystem should, at a minimum, contain the following 
six functions; 

o Task training requirements definition* 

o Objectives and tests development. 

o Instructional materials development. 

o Task training authoring system. 

o Instructional management and delivery control. 

o Job literacy interface. 

The training materials produced by this component would directly 
support Job-^site training by providing the means to train the taak 
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skllla and knowledge required for Individual task proficiency and 
certification* Air Force Instructional system development (ISD) policy 
could be operatlonall2ed In the Job-site environment through the 
development and operation of theae functions* Operatlonall2atlon of the 
Air Force ISD process should, at a minimum, provide the capability to 
perform the following: 

o Review knowledge^ skill, proficiency^ and Job literacy require-* 
ments asaoclated with each Job task for which training will be 
developed * 

o Catalog specific skill and knowledge performance/learning 
objectlvea essential to the development and maintenance of 
Inatructlonal materials and teata, so that requlrementa for 
revlslona to» and Improvement of » these materials can be 
readily Identified* 

o Develop effective and efficient Inatructlonal and testing 

strategies and tranaform them Into apeclflc Instructions for 
the ITS master Inatructlonal management and delivery control 
(MIMDC) data baae* These atrateglea would be used for making 
trainee assignments, evaluating trainee performance, and for 
gathering and reporting Information related to trainee progresa 
toward proficiency on a Job task* 

o Construct and catalog knowledge and skill test banks* 

o Catalog^ deliver^ and formatlvely evaluate all Instructional 
materials developed to aupport Job-site training^ Including 
thoae which are designed solely to Increase Job literacy* 

o Track OJT Instructional materials development progress through 
the major proceas stages* 

Development progresa can be tracked through the maate;c:>:task 
training requlrementa (MTTR) function deacrlbed earlier* Following the 
Identification of the task and a determination that training on the task 
la required^ a development agency would be dealgnated and the 
Identification and location of the agency would be entered In the IfTTR* 
Eatlmatea of the development manhours required for each development 
product or document would be made and Inserted In the MTTR to provide a 
basla for determining the progresa of the development effort* 

b* Training Delivery Component * Thla component provldea a means 
to assign all Inatructlonal actlvltlea and deliver all Instruction 
specified for certification on a Job taak and to provide acceas to all 
aystem functions In support of certlf Icatlon^ recertlf Icatlon, and the 
training quality control actlvltlea of ITS* In order for ITS to achieve 
this goal, a detailed set of structured apeclf Icatlona related co all of 
the Instructional activities required during the training proceas would 
be defined In the maater Instructional management and delivery control 
(MIMDC) data base during the training development procesa* Theae 
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controls as established in the HIHDC then become the primary mesns for 
the system to manage and control the delivery process from the time that 
a Job task is activated for training until certif ication/recertif icstion 
is achieved by the airman or the quality control external evaluation 
function is completed. Major functions provided in this component 
include assignment generation, msterials, media and devices, and 
interfaces with the training management and evaluation subsystems. 

1. Assignment Generation This function was designed to 
provide a controlling mechanism for initiating all instructional 
delivery activities that are performed within the system and to provide 
the necessary access controls that allow the trainee, trainer, and/ or 
supervisor to communicate with the system during the training process. 
This function would also generate instructional delivery assignments to 
each trainee in accordance with specifications established in the MIHDC 
and ensure that the assignments are compatible with other Job task 
assignment controls/prerequisites established at the Job task control 
level within the system. 

2. Materials - Instructional materials for ITS-supported Job- 
site training will need to be developed in a variety of forms 
corresponding to the specifications provided by the instructional 
strategy design and media selection process* Assignment generation will 
require providing references to the specific materials required/ 
available for each instructional activity identified in the master 
instructional management and delivery control data base. A package of 
materials for a task knowledge instructional activity should include, at 
a minimum: 

o A list of the learning objectives for each assignment 
within the instructional activity - 

o Any special instructions for the trainee on the use of the 
materials, such as how to obtain adjunctive resources - 

o The learner-paced instruction required for the attsinment 
of the objectives - 

o A self-test based on the learning objectives of each 
assignment - 

o A guide for self-*remediation, keying delf-*test items to the 
location of the correct information in the instructional 
material; e.g., page number, frame number* 

o Instructions for accessing the specific test to be taken 
upon completion of study - 

o Feedback statements to be generated by ITS for trainees, 
resulting from each input- 



Options available to developers for task knowledge Instruction 
Include the following types of materials: 



o Programmed text booklets* 

o Trainee workbooks using adjunctive materials such as TOs, 
AFRs, etc. 

o Learner-controlled slide/tape programs, 

o Single-concept films . 

o Audio tapes with adjunctive materials* 

o Tutorial guides for trainers . 

o Computer assisted instruction (CAI). 

To provide an acceptable degree of control of trainee progress, 
and better Integration of OJT Into the production environment, task 
knowledge assignments should he designed for a maximum study time of 
approximately 2 hours. Segmentation of Instructional activities will 
allow a training developer to achieve these goals. 

Options for materials to be provided for Instructional 
activities related to the practical training of a task Include the 
following: 

o Trainer guides and task breakdowns for demonstrations and 
practice activities , 

o Printed programmed Job task plans for trainees* 

o Slide/tape Job task demonstration sequences, 

o Single-concept films < 

o Simulation of field case problems provided by computer- 
driven or stand-alone devices • 

o Feedback statements to be generated by ITS for trainees and 
trainers as a result of each Input. 

3. Media and Devices - This function was designed to provide a 
means to employ multi-media Instructional materials, training devices, 
and any simulators, CAI devices, or other high technology training 
devices needed for delivery of Instruction during the training process 
and to define the requirements for Interfacing them electronically with 
the system when the capability exists. The assignment generation 
fimctlnn prevlouflly deacrtbcid fnr ntjaklng ;t*iitl|tnni(«nt. rt to ( ortt rur r- lonnl 
materials, mectla, and devlcea baaed apaclf Icatlona eatabllahed In Llit? 
KIMDC, slso la designed to generate a schedule demand for the resource. 
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Uhen a training resource In this component Is also electronically 
Interfaced directly with other functions of the system, a notification 
to establish a suspense for the assignment could be generated* The 
trainee would then sign on to the device and remain under the temporary 
control of the device until the designated Instructional delivery was 
completed* The results of the assignment would then be summarized by 
the device In the format specified by the MIMDC Instructional assignment 
and forwarded to the trainee evaluation function for determining the 
next assignment* 

This function of the delivery component also allows taking 
advantage of all scate^of-the-art Instructional presentation media* Of 
particular Interest Is the use of thoae media In computer-^asalated 
Instruction (CAT)* The Instructional material could be controlled In 
one of two ways and In either case provlalon must be made for the 
Interface of the CAT hardware to the ITS computer* First, CAT lesson 
presentation could be controlled by the ITS computer directly. In this 
case, the Interface must allow for high volume tranamlsslon between the 
CAI terminals and the ITS computer. Care must be exerclaed ao that the 
performance of the ITS support stations are not degraded by the CAI 
processing* Second, CAI lesson delivery could be controlled via a 
micro-computer Incorporated Into the CAI terminal. Iti this case the 
Instruction would reside external to the ITS computer on the micro- 
computer's storage medium. The micro-computer would also Incorporate 
the capability to bring varloua other Instructional delivery devices 
Into the learning process. Provision would also need to be made for the 
Interface of these devices with the ITS computer for the transmission of 
summary performance data to the ITS data base. The following generic 
types of Instructional delivery devices were envisioned for use In the 
ITS: 

o Lo\^cost black-and-white terminals without graphics 
capability * 

o Full-capablllty color graphlca terminals ^ 
o Simulators* 

o Stand-alone micro-computer devices- 
o Part-task trainers* 

4* Training Management and Evaluation Interfaces - This 
function was Incorporated to address the essential Interfaces which must 
exist In order to Initiate, conduct, and conclude the Instructional 
delivery process* Such Interfaces Include the following types of 
capabilities: 

o Software to enable delivery resources to be scheduled, 
allocated to groups and Individuals when required, and 
restored to "available for use*' status when the delivery 
function Is concluded* 
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o Trainee skill and knowledge testing to be conducted when 
appropriate in the learning process - 

o Storage and update of detailed performance data gathered 
during the delivery process •> 

o Reporting of the achievement of key milestones related to 
instructional activity status - 

o Appropriate notification of changes in task certification 
status to be made. 

OJT Evaluation Subsystem . The OJT evaluation subsystem of the ITS 
was designed to provide trhe means to assess trainee task knowledge and 
proficiency and to collect and analyze data related to the efficiency 
and effectiveness of the Air Force OJT system. In general, it would 
accomplish all of its functions by analysis of data generated within the 
overall system, either through its own quality control process or 
through other ITS subsystems and components. Components within this 
subsystem were identified as follows; 

o Trainee evaluation component 

o Training quality control component 

o System evaluation component. 

a. Trainee Evaluation Component . This component would provide the 
means to evaluate trainee performance and progress related to both 
specific tasks and duty position qualification. Performance data 
collected within the ITS to utilized as measures for trainee 
evaluation include assignment module-completion time data, trainee 
response data, remediation data, certification and decertification data, 
and training QC evaluation data. Knowledge testing would be based 
entirely on the prerequisite knowledge learning objectives for specific 
tasks; proficiency testing Would be based on the performance and 
proficiency objectives derived within the ITS training requirements 
"management'component . In addition to programmed evaluation checkpoints 
within the task training process as specified by the master instruc- 
tional management and delivery control (MIMDC) function, evaluation 
instruments for specific tasks performed in the work center would be 
accessible by supervisors for use in assessing airman proficiency on an 
impromptu basis. Further, all airmen would be subject to random task 
proficiency evaluations (on any tasks in the ITR on which the airman is 
currently ccrtif L.?d and is required to perform i!^ bis/her ^issigned duty 
position) conducted by the training QC section as part of the external 
evaluation function of the ITS. 

b. Training Quality Control Component » The purpose of this 
component is to provide systematic task proficiency evaluation 
procedures in order that the effectiveness of the training conducted 
within the OJT system may be assessed. It should b& emphasized that the 
primary intent of thi» quality control proccsn Is not to nsfseno the 
airman! but rather to aasess the quality of the training .that the airman 
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received and to correct any deficiencies detected. This training ayatem 
quality control process would consiat of three major steps) 

o Quality control pre-evaluation procedurea/activitiea: The 
process that randomly identifiea a taak and the airman to be 
evaluated^ selects a qualified third party evaluator to conduct 
the evaluation^ and schedules the evaluation* 

o Quality control external evaluation procesa: The proceas 
whereby the evaluation is actually conducted utilizing 
specif icstions defined for the task in the MIHDC. 

o Quality control poat-evaluation procedures/activitieat The 
proceaa through which the reaulta of the evaluation are 
reviewed and corrective action ia taken. 

c. Syatem Evaluation Component . This component would provide a 
means to analyze quality control^ training performance, and other data 
collected within the ITS to meaaure the quality of training accompliahed 
and evaluate the ef fectiveneas and efficiency of the training ayatem in 
meeting the requirements of OJT. Major functiona to be accompliahed in 
this component include the following: 

o Unit training analysis: The methodology for and production of 
a variety of management information suomtary reports identifying 
training accompliahed^ the status of the unit in achieving its 
training goals, and the ef fectiveneaa of unit training in 
meeting the requirementa of the mission. 

o Training system ef fectiveneas analyais: The methodology for 
acd production of a variety of management information auimnary 
reports identifying the degree to which the ITS meets the OJT 
system performance atandarda eatabliahed by the Mr Force. 

o Training ayatem coat analysis: The methodology for and 
production of a variety of management information summary 
reports identifying the coat of training at the job aite and 
the efficiency of unit training in meeting the requirements of 
the miasion . ^ 

o Training syatem capability/capacity annlyais: The methodology 
for and production of a variety of management information 
reports (a) concerning the capacity of a unit to conduct OJT ao 
that relationshipa between training load, training quality, and 
unit training effectiveness can be eatablished^ and (b) 
identifying deficienciea in training capability due to r^onr 
availability and/or inefficient utilization of training 
resources or lack of qualified trainers. 

Computer Support Subsystem . This aubayatem is eaaential for 
providing the various computer hardware and aoftware componenta of the 
system. It containa the hardware^ terminala^ communicationa linka, and 
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software required to sustain the OJT management, development and 
delivery, and evaluation subsystems. 



Personnel and Support Subsystem . This subsystem provides the means 
for specifying the personnel and organisational requirements for the 
overall operation of the system and providing the training packages 
needed to Indoctrinate the users of the system. It would also Include 
required logistics and maintenance functions. With respect to the ITS 
development effort. It would provide a plan for trar^^ltlonlng the system 
throughout the Air Force and ensure that system Implementation, 
operation, and expansion requirements are provided to the user. 
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V. DISCUSSION 



A research and development program of the scope suggested by the 
ITS functional specifications has implications for the Air Force related 
to job-site training and training management issues which hsve» in the 
past, been considered in isolation rather than in the context of the 
overall readiness posture of the Air Force. The definition of the 
functional requirements, and their organization into s system in which 
critical relationships between training and training management 
functions and the unit mission are identified, should i-ltimately provide 
the capability to respond more readily to readiness requirements. A 
more detailed view of some of the specific issues that will need to be 
addressed in the development and Air Force-wide implementation of the 
ITS, and a discussion of problem areas which are likely to be 
encountered, should be helpful to those managing the effort. These 
issues are grouped, insofar as possible, into critical R&D implicstions , 
essential elements of implementation, and long-^tetm impacts on other 
personnel programs. 



Critical RSD Issues 

The issues within this category are critical to reaching the objec- 
tives of the ITS, in that they deal with the major functions of defining 
valid individual training requirements, establishing measures of 
training effectiveness, and controlling the quality of training, all of 
which are deficient in the current OJT system. The ITS could not 
function to standardize training and evaluation, nor could user 
acceptance of the training system be gained, should the Air Force decide 
to relegate the decisions involved in resolving these issues solely to 
work center production personnel. An appropriate level of RfiD which 
directly addresses these issues and provides methodologies that can work 
as a part of the training system, is essential to the long-^term success 
of the ITS and the OJT system which it will support. 

The ITS training requirements management component will have, in 
some degree, the following structure. First, it is envisioned that a 
master task list (KTL) will be constructed for a specialty. This list 
would be a standardized source of tasks from which both generic ^nd 
specific position task lists can be constructed. Second, it is also 
envisioned that generic position training requirements (6PTR) task lists 
would be constructed from the MTL. Here a ^'position*' refers to an 
individual's job. A GPTR must be a list that includes "important** tasks 
that are common to a group of similar individual duty positions. 
Finally, an operational position training requirements (OPTR) task list 
would list the tasks for a particular unique duty position* An OPTR 
would serve as the basis for identifying, controlling, and monitoring an 
individual's training on his or her current Job. It will not 
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neceasarlly be the case, at least at the outset , that these various task 
lists can be constructed in the sequential order in which they are 
described here. 



^^TL Construction 

In the context of the ITS, the major purpose of an MTL is to 
provide standardization of tasks across specific individual duty 
positions (a duty position ia the work done by exactly one individual) 
and, insofar as possible, across generic positions or job types as well. 
At the same time, task descriptions should be sufficiently specific that 
activities whose differences are important for training are, in fact, 
different tasks. Finally, it is important that MTLs have a simple 
relationship to those used in other Air Force data bases — particulsrly 
that of the occupational survey prograia. In short, tasks must not be 
too specific, because of loss of standardization, as well as for 
practical reasons. Nor should they be too general, because extremely 
broad tasks are not likely to be useful in identifying and controlling 
training requirements and job performance skills. Determining the 
appropriate level of specificity is, as may be Reen, an important issue 
which must be addressed in constructing MTLs. 

As mentioned above, another important consideration in constructing 
tasks for an MTL is comparability to other data bases. Perhaps the most 
salient of these data bases is th^t of the Air Force occupational survey 
program. Occupational surveys provide much information which is used in 
training-related activities, such as determining job classification 
structure, aptitude requirements for various career fields, and formal 
training requirements. Task data routinely gathered in occupational 
surveys include percentages of job incumbents performing tasks, relative 
time spent in performance of tasks, learning difficulty, and recommended 
training emphasis for first-^term airmen. In order to ensure 
compatibility of the ITS with Air Force job classification, personnel 
acquisition, formal training, etc. and in order to use occupational 
Survey data in the ITS to manage training requirements, MTL tasks should 
have a simple relationship to occupational survey (OS) tasks. Ideally, 
the relatioTXship might be one^to^one. This would imply that the maximum 
number of MTL tasks be less than 1700 (the maximum number of tasks that 
can be handled by most Comprehensive Occupational Data Analysis Programs 
[CQDAP]). The practical limit to the number of tasks per specialty 
which can be handled by OS technology is approximately 1200 to 1300. 
Further, this implies that OS task lists could be the primary or, ^t 
least, the major source of tasks for ^n MTL. 

Another question is whether OS-type tasks are sufficiently specific 
for ITS purposes, qs data have been used for years in determining 
requirements for formal resident training provided by the Air Training 
Command (ATC)* This would seem to suggest that OS-type tasks are 
sufficiently precise. However, OS task data are not used in isolation— 
they can be and often are broken down according to equipment used, job 
type '^groups of specific positions which are homogeneous with respect to 
tasks performed— analogous to generic positions) and many other relevant 
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variables* This suggests a way of achlavlng greater task specificity In 
the ITS while maintaining compatibility between MTLb and OS task lists. 
OS tasks could otake up an MIL, while sddltlonsl codes could be contained 
In the ITS Identifying relevant variables for Identifying subgroups of 
airmen for whom particular tasks differ In content — say, types of 
equipment used or job type (generic position) groupings. 

In some speclaltlet), other task-oriented data bases are available. 
A major example In malntenance^orlented specialties Is the Logistics 
Composite Model (LCOM) data base. As with OS data, data from sources 
such as LCOM may be useful In the ITS. Such data bases laay also serve 
as sources of MTL tasks* Thus, It Is Important to Identify such task 
data sources and to achieve some compatibility between MTLa and such 
data bases. However, these data bases are not necessarily compatible 
with the OS data base either In tasks used or In types or formats of 
data available. Thus, It may not be possible to achieve complete 
compatibility. In cases such as these. It will be necessary to decide 
which compatibility Is most Important. Consideration In this decision 
should Include numbers and types of tasks, types of data available, and 
the usefulness of such data In the ITS; compatibility of the ITS with 
other declslon-^maklng based on such data; and standardisation within the 
ITS. This last point deserves further contment. Information from most 
of these data bases Is available only for certain job specialties; for 
example, LCOM data are available only for maintenance specialties. OS 
data are available for most specialties, thus, one of the trade-offs 
that should be considered In the ITS design Is between having different 
procedures for different specialties depending on the types of data 
available and having common procedures for most or all specialties using 
data which are usually or always available. Even OS data are not 
available for all specialties, thus. It Is advisable that MTL files (as 
well as all of the ITS) be structured not to depend on any single data 
source. The other side of this Issue Is that OS or other data bases 
could be modified to meet ITS needs. 

In constructing MTLs, one should start with existing tasks from OS 
data or other sources as appropriate. To ensure compatibility, tasks 
from such sources should be modified as little as possible. Hovever^ 
provisions should also be made for tasks to be added that emerge 'from 
elsewhere In the ITS — particularly from OFTR task lists. 

Identifying Generic Position Training Requirements 

GPTRs are Intended to be lists of tasks which are common to groups 
of similar specific duty positions. GPTRs are also sources of tasks for 
OFTRs, as well as a means for monitoring and controlling specific 
position training. 

The first step In constructing GPTRs Involves Identifying generic 
positions. Two general approaches might be used for this purpose. The 
first Is an a priori approach. In this approach, generic positions 
would Identified through a rational analysis of how work la structured. 
In specialties such as Security Police In which a particular 
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exists that Job types are appropriate). This problem can easily be 
remedied through the CODAP cluster analysis methods used in OS. As 
mentioned above , a hierarchical clustering method would be used« tn 
this method, a sequential series of cluster solutions are obtained, 
starting with the case in which each incumbent forms his or her own 
cluster and becoming more general or aggregated until the final solution 
consists of one big cluster containing all Job incumbents « This means 
that clusters^ Job types, or generic positions of any desired homo- 
geneity can be found among the various hierarchical solutions obtained 
(provided that clusters of the desired homogeneity actually exist among 
the Job incumbents). The point is that even if current OS Job types are 
not at an appropriate level of specificity for the ITS design^ OS 
analysis procedures could probably be modified to produce appropriate 
generic positions. 

Once generic positions have been identified^ GPTRa need to be con~ 
structed for the generic positions* Given identified generic positions 
and task performance data for incumbents in the generic positions^ the 
problem would be to select tasks which are "important** across specific 
positions within a generic position. Two issues arise here — how to 
measure the importance" of tasks to generic positions and how 
'^important** a task must be to a generic position before it is included 
on that generic position's GPTR. One important factor in the 

importance** of tasks within generic positions is how widespread 
performance of the task is among incumbents. This might be measured by 
the percentage of position members who perform the task^ or the amount 
of time spent on the task by position members. Another factor in the 
"importance** of tasks in positions for ITS purposes is task difficulty* 
Extremely easy tasks probably do not need to be put on GPTRs because 
training is not normally an issue in such tasks. Other task 
characteristics^ such as various criticality measures^ may also be 
important. Task performance (percent members performing and percent 
time spent) and task difficulty data are routinely gathered in the OS 
program. Other task characteristics could be gathered via OS technology 
or by other means. The questions confronting ITS developers concern 
what task characteristics should be measured^ how to measure them if 
ftuch data are not now routinely gathered, and how to combine them into 
an overall selection criterion variable for including tasks in (?TRs. 

Given an overall **importance** score^ or at leasts scores on several 
relevart variables^ procedures will need to be developed for deciding 
wiich tasks will go onto a GPTR. The main issue here concerns whether 
all tasks should be included on a GPTR which might go onto QPTRs for at 
least a few airmen in a few positions or whether only those tasks which 
are relevant to most or all airmen in a generic position should be 
included. This decision will^ in turn^ depend to a great extent on the 
degree to which GPTRs conatrain supervisors in task selection versus 
simply serving as a convenient source of tasks for OPTRs. 
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Constructing Operational Pogltlon Training Requirements Task Lists 

A critical issue in constructing OPTRs concerns the degree to which 
supervisors will be constrained by the system first, in selecting tasks 
to go on OPTRs and secondly, by the tasks on their OPTRs once 
constructed. At one extreme^ specific positions could be assigned to 
generic positions by the system; OPTRs would be Identical to 
corresponding GPTRs and supervisors would be responsible to train on all 
tasks appearing on an incumbent's OPTR (which is equivalent to the GPTR 
in this case). In this arrangement, no opportunity would exist for 
local iQodif icatlon of OPTRs or of actual tasks trained. At the other 
extreme, the HTL and GPTRs would exist solely as a source of tasks for 
locally constructed OPTRs. Supervisors would be free to put any tasks 
desired on OPTRs and to provide any level of training desired on OPTR 
tasks. It is unlikely that either of these extreme cases would be used 
in practice, although much local flexibility should be allowed. In any 
case, the degree to which **the system" is prescriptive concerning OPTR 
content and training will have a significant impact on procedures for 
constructing OPTRs, as well as for constructing GPTRs and ^fTLs. 

Four major steps will probably be required in constructing OPTRs. 
First, each operational position must be identified, if possible, by 
selecting the closest generic position match. That could be done semi- 
automatically by having incumbents (or perhaps supervisors, keeping in 
mind that supervisors generally have less knowledge than Incumbents of 
tasks actually performed) Indicate tasks that are in the operational 
position, and then using statistical means to Identify the generic 
position most similar to the operational position. Another approach 
would be to provide supervisors with all GPTRs and let them select 
generic positions for their specific positions. 

The second major step in constructing OPTRs (the stepa need not be 
done in this order) is to select tasks from the GPTR of the generic 
positilon to which an operational position has been assigned. The main 
issues here concern guidelines and data about tasks which need to be 
provided and used for this purpose~ln particular, the degree to which 
OPTRs are constrained to Include tasks from assigned GPTRs. 

The third step concerns selection of other tasks from the MTL to go 
on OPTRs. Again, an impoxtant issue concerns the degree to which OPTRs 
are constrained to include or not Include such tasks from an MTL that 
are not on a position's GPTR» 

The last atep Involves adding to OPTRs tasks which appear neither 
on a position's GPTR nor on that specialty's MTL. Several Issues appear 
to be important. First, it seems desirable to encourage use of tasks 
from the MTL and GPTR rather than using locally written tasks, where 
possible. Yet it also seems necessary to provide for use or input of 
local tasks. Given the possibility for local tasks, some form of 
quality control will be needed for constructing task descriptions for 
such tasks and for evaluating training on such tasks. Also, provision 
should be made for local tasks to be placed on MTLs and GPTRs if it 
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turne out that euch local tasks are really more widely performed. This 
would involve collecting local taeke st some central location^ screening 
them^ and feeding them back into the eyeteia. Screening would be 
required because the same task may appear in somewhat different form in 
different locations when tasks are locally developed. Provision is 
needed to identify such situations and to construct a task statement 
that will be acceptable to all relevant locations. 

Training Effectiveness Index 

A potential feature of the ITS is its ability to provide measures 
of training effectiveness or training status. This could be done both 
for individual job incumbents and for entire organizations. 

Consider^ first ^ training status measurement for one person. The 
first step involves measuring a person's performance on all tasks on 
that person* s OFTR. Ideally^ this would be done by actual performance 
tests. In practice^ construction and administration of performrmce 
tests for all tasks and job incumbents may not be practical. K 
reasonable alternative involves relying on supervisor or OJT trainer 
evaluations of task performance^ which would be audited by performance 
testing on sample tasks ^.izi incumbents* Construction of these 
performance evaluations is not a simple matter. In some respects^ a 
go/no go approach may simplify evaluation^ but^ particularly in actual 
performance testing^ some complications arise as well. A typical 
performance test measures level of performance^ and a go/ no go system 
requires that it be decided in advance what level of performance is 
sufficiently good to be considered "go.** In other words^ minimum 
passing scores need to be set (at least implicitly) for such tests. 

Given "can perf orm**/"cannot perform** measures for all tasks 
(including locally developed tasks)^ the next issue concerns how to 
aggregate these measures into an overall performance index for an 
individual job incumbent. An obvious approach would be to compute the 
proportion of OFTR tasks that an incumbent has been certified to 
perform. With this type of measure and supervisor emphasis on achieving 
high training effectiveness scores for their subordinates^ it would be 
very tempting for supervisors to put as few tasks as possible on OFTRs* 
This would subvert the objectives of the system. Fositive incentives in 
some form for people to put all ''important** tasks on OFTRs and to 
conduct honest performance appraisals should be included in the ITS. In 
fact^ positive incentives for people to not "game the system** are a 
critical aspect of the ITS. In overall performance measurement^ one way 
to encourage supervisors to put appropriate tasks on OFTRs might be to 
use a measure in which that percentage of tasks certified is weighted by 
the number of tasks on an OFTR^ rather than the simple proportion of 
OFTR tasks that have been certified. In such a measure^ lower 
percentages of task performance would be compensated for by having more 
tasks on an OFTR^ encouraging supervisors to put tasks on OFTRs even 
though more tasks might reduce the percentages that can be performed at 
a given point in time. 
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It is probably the case that some tasks are more "^important" or 
"critical** to overall successful mission performance of an organization. 
This suggests that, rather than just using percentages of OPTR tasks 
that can be performed, a more accurate picture of an organization's true 
training status would be obtained if tasks were weighted by criticality 
in an overall performance measure. This would require that measures of 
task criticality be available. Occupational survey R&D re&ults indicate 
that measuring taak criticality presents some difficulties. First, one 
must very carefully define exactly what i& meant by criticality. 
Secondly, it appears that criticality cannot be measured as a single 
task characteristic, at least by OS task factor measurement procedures. 
In&tead, it is necessary to decompose criticality into several task 
characteristics. Any one of these characteristics may or may not be 
relevant in a particular specialty. In the OS training emphasis R&D, 
criticality was decomposed into consequences of inadequate performance 
(How bad ia it if the task is incorrectly performed?) and task delay 
tolerance (If you find out right now that you need to do a task, how 
long do you typically have before the task must actually be done?). In 
some specialties, little variation has been found among tasks on these 
factors. In other words, all tasks were about equally critical, as 
measured by these factors* The implications of this for criticality 
measurement in the ITS are that several task characteristics will 
probably need to be measured and combined into one overall index. 

Another issue in task criticality measurement concerns comparison 
across jobs and specialties. Typically, aa in OS methodology, criti- 
cality task characteristics are measured by having subjects-matter 
experts — people familiar with the tasks and jobs containing the tasks — 
rate the tasks. Such subject-matter experts usually can rate only tasks 
in one job specialty because they are not sufficiently familiar with 
more than one specialty. Thus, task criticality ratings cannot be 
compared across specialities. However, in the ITS one would want to be 
able to compare training effectiveness across job specialtiea. This is 
particularly true for generating organization- level effectiveness ' 
measurea, aa will be seen below. For ITS purposes, not only is there a 
need to develop task criticality measures, there is also a need to 
gather those data so that the values are comparable across specialties. 
At AFHRL, this problem has been termed the "benchmark" problem. At 
least two approaches have been tried for gathering benchmarked task 
data* The first is that used in the aptitude requirements R&D, in which 
expert judges were given an opportunity to study and observe selected 
seta of tasks in aeveral specialties and eventually to rate those tasks* 
From benchmark task ratinga on these subsets of tasks, relative ratings 
of other tasks can be translated into the benchmark scale. The other 
approach is essentially statistical in nature and has been used to 
benchmark atrength and atamina task ratings. 

Once individual^ level training effectiveness measures have been 
constructed, it would be useful to have organization- level measures. In 
order to do this, a meane of combining individual- level scores Into a 
single organization score must be dsvsloped. Since an organization 
typically contains people in several specialtiea, to compare 




organizations with different combinations of specialties requires that 
individual- level scores be comparable across specialties— the benchmark 
problem. Organization- level measurement may also require that 
organization task or mission criticality be measured~a whole additional 
issue for investigation* 

Input to OS from ITS 

Much of this discussion has concerned ways in which the OS 
methodology and data base can support the ITS. The ITS can also support 
the OS system. Some of the possibilities are discussed here. 

If the ITS MTLs are compatible with OS technology and if procedures 
exist in the ITS for locally developed tasks to be ente.*ed into the 
overall system, then the ITS can provide a continuously updated source 
of task lists for OS studies. While the USAF Occupational Measurement 
Center (OMC) has a program in place today to update LasLc lists 
continuously, that might be accomplished in a more thorough or efficient 
manner via the ITS. 

Assuming ITS tasks will be compatible with OS tasks, the ITS can 
provide task*level data that would be useful in Job analysis. One 
example of this is time-to-train data. These time-to-train data will 
probably not be available for all tasks, but data of that type for sonke 
or most tasks would be useful. Task*level performance ability data are 
also potentially very useful. One application of those data would be in 
developing and validating personnel selection and assigrtment procedures. 
Clearly, OJT training status data would also be relevant for determining 
requirements for and effectiveness of formal training—an important 
application of the OS process. 

The ITS could serve as a means of gathering other OS data— both 
task performance data and task factor characteristics data. This could 
allow the OS data base to be updated on an almost continuous basis. 

Training Quality Control 

The training quality control functions of ITS are intended to 
provide the data required to make accurate assessments of the degree to 
which the training meets Air Force mission requirements. Some R&O 
considerations in deriving training effectiveness indices for individual 
airmen and organizations have been identified above; the appropriate 
external mechanisms for their validation, and the detection and 
correction of training problems at both the work center and the system 
levels is yet another area for major RSD support. One issue to be 
resolved concerns task performance evaluation. The task and airman 
selection algorithms devised must provide assurance that sample sizes 
are adequate to provide valid training assessment Jata. Secondly, and 
closely associated with this, is the efficiency to be gained if task 
performance can be evaluated reliably using mechanisms other than 
direct, third-party evaluations and, if not, the effectiveness of 
critical part-task evaluation procedures* Finally, and not the least 
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important, is the identification of the significant factors affecting 
user acceptance of the external evaluation process and results* 



Implementation Issues 

This category of ITS transition issues concerns important elements 
of the system concepts and functions which could receive too little 
emphasis from managers and others involved with the program. Most of 
the elements of the ITS have user acceptance implications, and this 
should be considered to be of major significsnce in ITS transition to 
Air Force"-wide use. 

The role of supervisors and trainers within the present OJT system 
includes the functions of determining Job training requirements; 
planning and scheduling the training; delivery and tracking of training; 
assessing the quality of training; and recording and reporting training 
accomplished. This role will change significantly with ITS 
implementation. In most work centers, the new role will involve the 
following of procedures and guidelines for task training pre-established 
by others, and providing data related to training the task to the 
system. One might expect that such a role change would be welcomed. 
However, most ITS management and evaluation functions will rely heavily 
on receiving accurate and timely data. This requirement could be 
perceived as an increase in data input functions to be performed by 
trainers and supervisors, suggesting a compromise to dilute or 
circumvent essential controls and functions of the system. The 
resultant fluctuations in both training quality and individual or unit 
training evaluation capabilities could adversely affect the acceptance 
of the ITS at the base level. Associated with this aspect of user 
acceptance within the work center, the Air Force should ensure that the 
system design includes a sufficient number of user computer terminals to 
eliminate manual recordkeeping. 

A second essential element of transition to Air Force-wide use of 
the ITS involves the need for strict observation on the part of training 
developers of controls on instructional and trainee management. A 
tendency of developers to transfer responsibility for these controls to 
trainers and supervisors rather than building them into the 
instructional strategies could develop, resulting in negative effects on 
training quality and standardization. 

Issues concerning the development and maintenance of large quanti- 
ties of Job-site instructional materials and instructional management 
controls constitute a difficult problem in ITS implementation. Included 
in this problem is the establishment or designation of an organization 
which can provide sufficient standardization of training development 
processes. This will involve coordination of training development with 
AFS functional managers and AFS users to resolve problems such as: 
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development prioritization within and across functional areas; 
availability oC subject-matter expert (SME) support; ways In which to 
adjust training and evaluation to compensate for local varlatlona In 
task performance requirements for the same task; specific-task training 
approval and Implementation decisions; and allocation of task training 
materials and equipment maintenance responsibilities* 

Closely associated with these training development Issues Is one 
which concerns the distribution of functions at the HAJCOM/SOA and 
higher level agencies which are related to base-^level training 
management for AFSs. These functions Include task performance standards 
review and approval; training revision requlrenents Identification, 
prioritization, and coordination; MAJCOM transition planning and 
tracking; distribution of training materials; and coordination of Job- 
site training evaluation for the correction of deficiencies. A similar 
aet of new functions related to MAJCOM/SOA AFS users will exist for 
weapon system, equipment, and functional area-specific training. The 
Air Force will have to assign responsibility for these new functions at 
levels commensurate with efficient and effective management and 
budgetary control. 

The assignment of management responsibility for training at each 
base Is essential to ensuring the success of the system. The functional 
responsibilities reflected In Table 4 describe many requirements that 
must be satisfied In an appropriate organizational structure that 
considers mission priorities. The operational organization for training 
syst^ management of all base-^level training will need a structure which 
can effectively distribute responsibility and authority for training 
quality control, training cost control, management Information 
Interfaces, training resource management. Instructional management, and 
training delivery. 

While the functions above relate directly to management 
responsibilities, each unit OJT manager should play an Important role as 
part of the base-^level pool of training expertise. Training quality 
control functions, for example, should be shared by all OJT managers, 
particularly assisting In trainee evaluation and maintaining 
coordination between training development, delivery, and system 
evaluation functions. On the other hand, the unit manager should be 
geographically located, for the most part. In the unit or squadron which 
he or she services. This would help to ensure that the ITS Is 
responsive to the udlque training requirements at the work center/ 
section levels. 

It Is with these considerations in mind that the current base-level 
OJT management responsibilities were reviewed In conjunction with the 
factors considered In defining OJT system requirements. An effort was 
made to determine the scope of organizational changes that may be 
required to support management of a new OJT system effectively. 

The current base-level OJT management function Is assigned as a 
subordinate unit to the CBPO un<^?r the Directorate of personnel. 



89 



96 



Placement of OJT management reBponslbllltleB at the base personnel level 
1b not considered to be the best organizational alternstlve for the 
following reasons ; 

o The Directorate of Personnel Is on the same authority line as 
agencies where most job-site training occurs. This situation 
currently creates difficulty In Implementing training policies 
and procedurea. Theae problems could be magnified considering 
that the ITS would Incorporate qualification training 
management functlona that Increase the need for more direction 
and visibility than Is currently provided. 

o Work center production, In most specialties, Is dependent on a 
sound job-site training program. While management of job-site 
training Is currently a personnel function, the responsibility 
for assuring work center production Is not a base personnel 
function. 

o When task proficiency Is not demonstrated after training 
certification, It Is not a base personnel problemt but a 
mission capability problem. 

o When changes occur In mission priorities^ weapons systems^ and 
workloads, or when contingencies arise affecting training, they 
are not base personnel problems. Such problems are mission 
responsiveness problems. 

o When training Is not efficient or cost effective^ It Is not 

just a base personnel problem. It Is a broader mission-related 
problem. 

o Job-site training relies heavily on the use and availability of 
operational equipment. Shortages and mlsallocatlon of such 
equipment, when they affect training for the mission, are not 
personnel problems, but are training resource management 
problems . 

o Training accomplished In support of mission readiness Is a 24*- 
hour-per*-day, three-shift operation, whereas base personnel Is 
typically an 8-hour-per-^day , one-shift operation. ?or this and 
other reasons, the base personnel organization Is often 
considered by production-oriented work centers to be an 
*'outslde activity** relative to job-site training. 

It Is therefore concluded that Improved training could be achieved 
by a separation of personnel and job-site training management functions 
In order to elevate training management to an authority level 
commensurate with responsibility for base^lde mission accomplishment. 
It Is recognized that some personnel programs rely on data provided via 
the OJT system. This support should continue to be provided from the 
ITS. 
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Long Range Impllcatlona for Other Manpowar 
and Personnel Functions 



The current OJT system provides data used directly or Indirectly to 
Implement other personnel policies; e.g., classification policies 
establish prerequisites for avard of APSC skill levels, which Include 
knowledge and qualification requirements. The classification system 
relies directly on the OJT system to provide the training needed to 
satisfy these particular prerequisites at the specialist and technician 
levels. Promo':lon policies are Indirectly affected, In that award of a 
particular skill level Is a prerequisite for promotion to a specific 
grade; e.g., 5-sklll level for E-4; 7-sklll level for E-6. 

No change In the relationship of OJT to other personnel functions 
Is envisioned as fi result of Implementation of the ITS. However, the 
types of data provided via the ITS may Invite changes to policies or the 
ways In which these policies are Implemented. 

Classification/Utilization 

APR 39-1, Airman Classification Regulation, contains within certain 
specialty descriptions the requirements to perform specific tasks prior 
to award of an APSC skill level. If a trainee has not previously 
performed and attained qualification on these tasks, the ITS could 
Identify these as training requirements. If a trainee Is assigned to a 
position where these tasks are not performed, utilization policies and 
procedures should support rotation to a position where performance and 
qualification requirements can be satisfied. For example, AFR 39*1 
requires personnel In AFSC 431X1 to attain hands-on aircraft maintenance 
experience, but some trainees are assigned to Job Control or Technical 
Order Publications Librarian positions In which actual maintenance Is 
not performed. These trainees need position rotation to obtain the 
required experience. 

The data to be provided by the ITS relevant to qualifications of 
personnel could present opportunities for Improved assignment selection 
procedures when specific qualifications are required to perform In 
critical duty positions. This would probably be most beneficial when 
short-notice, contingency/emergency operations must be activated or when 
one-deep positions must be manned. 

Quality Force/Promotions 

The ITS would provide capabilities for assessing Individual 
qualifications and progression of training on a task-by-task basis. The 
availability of these data could Influence ratings on Airman Performance 
Reports (APRs), particularly within the area of "training responsibili- 
ties.*' Since the Weighted Airman Promotion System (WAFS) considers AFR 
ratings in the promotion selection process, greater opportunity for 
selecting the most qualified persons may be presented. 
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Manpower 

Studies conducted by Management Engineering Teams (METs) to measure 
manpower requirements may be enhanced through the availability of data 
relevant to specific tasks performed within functional areas and to time 
required to train/perform tasks within work centers and positions. 
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VI * GOMGLUSIONS 



There Is great potential for Increasing both the effectiveness dnd 
efficiency of mleelon-orlented training by the application of computer 
technology to the functions of InBtructlonal management » scheduling, 
reporting, external evaluation^ and recordkeeping. However, such 
support provided for those functions should be accompanied by guidance 
and capabilities which reaffirm the Air Force commitment to the 
principles of Instructional Syetem Development for all training^ 
Including taok-orlented training. This guidance should clarify the 
relatlonBhlpB between the componentB of the training system and the job. 
Tasks must be Identified In more Bpeclflc terms and keyed to jobB to 
allow more positive Identification of training requirements for thoBe 
Jobs. RequlrementB for task performance must be derived from Bpeclflc 
duty position requirements and not be establlBhed merely to conform to a 
hierarchy of skill levels. Training must be administered^ managed^ and 
delivered In modes which consider Individual differences In ability. 
Perhaps most Importantly, taBk- and mlsslon^orlented training muBt be 
evaluated using appropriate measures* It Is In training Bettings with 
these characteristics that compute r-baBed technology can provide the 
management and administrative functions that will enBure that airmen can 
perform with required proficiency thoBe tasks for which they are 
trained. 
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POSITION SURVEY 



OAFSCt POSITIONi 

PAFSCt COHHERCIAL PHONE NOt 

2 AFSCt 

NAHE: _ _ _ RANki. 



MAJOR COHHANO/ AGENCY OJT FUNCTION 
O MAC □ HAJCOH OJT STAFF 

C3 SAC CZI HAJCOH FONCTIOMAL MANAGER 

□ O CBPO OJT STAFF 

CZI OTHER: C3 COMMANDER 

O WING TRAINING MANAGER 

O SOoN/UNiT OJT MANAGER 

O SOoN/UNIT TRAINING COORDINATOR 

O WORKCSNTER SUPERVISOR 

O IMMEDIATE SUPERVISOR 

O TRAINER 

CD OTHER; 

OKGANlt^TlON: OFFICE CODE: 

ON-BASE 

BASE; ^ LOCATION: 



ATTACHMEt4TS: 

^3 Recotd SuntTtiaty Sheets 
CD R«pot t SumitiAr y Sh«ets 

Fot» Summary Sheets 
n Training Summary Sh«e tfi 
^3 CDC Surveys 

O Traini>e Confi ider a t ions Surveys 
^3 Con tlnu&tlon Sh^rets 
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iiiW TtuiitxfHj lKv«r)upnirht h Delivi'ry Sut^^ystrm - Tdftk Proficiency Objectives 

Dl.l Tho CJT pt^t^tn h^s u'^en criticised fot rtQi being refiponsivc* tQ 

ii>i£siL>(> reijuiremontiri !n yoor opmiori^ how could OJT, i^r t it^ul^r ly it\ 
ihi: tdsk proficiency AteA, be feoriented tow^td mission rei^uirements? 



Dl*2 Where are the conditions and standards now specified for JPG task 
performance? 



Dl,3 Who should develop tafik proficiency objectives? 



Dl*4 Cotild occupational survey data be collected which could be utilized 
to formulate task proficiency guidelines? 



Dl^S Could task proficiency objectives be derived from any other source? 



D1.6 How specifically should task proficiency objectives be stated 
(behavior^ conditions^ standard^)? 



Dl.? How many times Should a trainee perform a task to be signed off as 
proficient? 



Dl^e Can task proficiency objectives vary for the same task as a function 
of changing positions? 
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1 



^ ^D2) Training Oev 



^lopmont < Delivery Subsyste,. - Job Rot^iort Objective 



D2 



1 «h.t t. the AF policy regaiding job rotation withirt specific 



functional ^icAs/specialt ies? 



What opportunities for Job 
weapons system? 



rotation are available within the hAJCOM/ 



02^3 HOW frequent 



frequently doe. planned ,ob .ot.tion unplann^.? 



,ener,I constraint. «e applicable to the i^b rotation policy, 

5 HOW doe. the 3ob rotation pclicy/standard affect qualification 
training? 



X X 



D2 



. .re t..re any specific constraints applicable to t.e ATSC. selected 
for the initial ITS design? 



.bat does t.e con«Pt of ^ob rotation i^I. for ITS, C«., etc.. 
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lt>3) Tr*inifi9 Development k [>elivci:y Subsystem - Ir)terfdC«# 
Occopat ion^l Sue v^y 



d3^1 In your opinion^ is occupational survey data being translated 
effectively into OJT training requirements? 



Hou can occupational survey methodology be expanded to make it inore 
responsive to the problem of defining training requirements? 



P3^3 Should occupational survey data eerve as a control for the: 
JfPC? 
STS? 

Task^by-position data base planned for ITS? 



What are the strengths^ or advantages of using occupational survey 
dAta to for;nuIate OJT training r«.qulremer)ts? 



What are the disadvantages/weaknesses of the concept of using 
occupational survey data to formulate OJT training requirements? 



XX 



P3,6' For what purpose should occupational survey data be used if not to 
determine training requirements? 
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UJT I 



Coold ITS generate a task list for occupational surveys? 



D3.e Could ITS se 



as a validation t<»l for occupational >. -rveys? 



XX 



r,3 9 Who should be responsible fcr in,plen,ent ing into the OJT systen, the 
results of occupational survey data? 
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iD$} T(4itMnq Oo;/elopm(^nt & DeUvety Subsystem ^ interface^ STS 



04.1 Is the STS as currently developed^ in sufficient depth ot detail 
to be utilized for ^11 present or potential functions the STS7 



04*2 Does the current proficiency code key adequately de^scribe to you 
3n individual's actual qualification for performance on the job? 
What are the deficiencies? 



D4*3 Should the STS reflect general tashs applicable to all ATSCs? 
ti*e*# administrative, managements supervision^ safety^ etc.) 



04.4 What sepatation should exist between the functions of the STS and 
the JPG? {i*e.f should STS be oriented towatd careet /advancement 
wheteas the JPG is oriented to a position/job/specific task?) 



04. & WhAt ate your perceptions on ptoviding a note clear-^ut division 
between cateer knowledge tSTS^ CDCs) and iob knowledge (specific 
instructional materials)? What would be the affects on pronotion 
and retention? 



XX 



04.6 What should be the function of ^n STS in a training syeten which provides 
only Job/task knowledge andproficiency, rtot career knowledge? 



D4*7. Who should be OPR (or an STS used only for cateet development? 
an STS used for both career development and task proficiency 
progression? 



Fot 



04«B What source document(s) should be used to cteate an STS? 
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t05) Tfdining Development 4 Delivery Subsystem - Training Aids 



D5.1 Wt\^t insttuctional technology is employ«d within th« HAJCOM for 
QJT and ancillary training? Who develops? tlho funds? 



D5.2 What instructi£>nal t^hf>olo>gy is employed at this base/unit? 



D5.3 Describe the specific uses of instructional technology at this 
location, {checkout, check in i testing , evaluat ion/validation) 



Are thete problems in maintaining an adequate inventory of multi- 
media materials? Trainin/9 devices? 



OS. 5 Are there allocation problejns when the deraand for nulti^media 
materials is high? 
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XX 



{D6 ) Tiaining Dovelotiment 4 Delivery Subfiyfitem ^ Ta«k by position d^ta base 

D6*l WhoVe are jobs and/ot positions on this bASt/sqdn/unit identified 
and approved? 



D6*2 Are positions basdd on a standard organisation? 



Ho«» many different AFSCs are repref.ated on this base? 



I>6*4 How many diffefent positions are established at this base? 



D6*S How many tasks are defined in this vork center? Is there a 
master JPG? 



D6.6 Who should build the task ^ position data base? 



D6*? Who should maintain the task by position data base? 



Who should be able to access the t^tsk by position data baser 

For inquiries? 

For update purposes? 



D6*9 What are yoor preferences concerning vhere and when the JPG should 
be generated? 
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4D7) T(Ainin9 Developinent aod Delivery 5ubsyst«Rt Ttainets 



07.1 Should th* ttairtet fot e^ch t^sk b« identified iti the ttaioee's 
training record? 



D7.2 Whdt qualificatiortA should a trainer ifteet? 



D7*3 «ow should the certification of trainers be document«a? 
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(0-Al) Sin*ul*tion of Tdsks Not Found on Job 



D-Al.l 



• ^ 1. L^hi,-h ace the responsibility of poiitionc tn thifi 

tcainea? (Contingency, mobility, etc,) 



D-Al.2 Which Skills related to tasks pecfocmed 
b« gained thcough simulation? 



in 



this workcenter could 
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n 



^^^V^J*^'!^ Treading Training 



« , , «h*t Dcrcent of PerBonnel in this woiKcenter have been 

^^llTin'the ELe riding i«^..ve.ent pro.r..7 



Ha« this P.09r-™ i«*-ov.d c.n^rehensioo of job-related reading 
water iaU? Uf not, why not?) 



—1 



U4 
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Ml. 2 ^ho *'nti^is trjluots in UGT7 

Hii3 Should iUiS I'xS diJ9n(jsi& oC training needed occur prior to 

JCi:^ i(ji^iiivnt , wht^n th<^ diiman icportti to th^ base^ oi wh<^f) the 
dLtmjn tcports to his unit? ' 

MU$ Whdt ba^clcvcl training should not be considered in the ITS 
dijfjnri&i^ of needs? 

Oticntcition? 
Ancillary? 

tvho should be responsible for initiating the ITS diagnosis? 

MU7 It th« ITS diagnosis is automate^i, should the results be 
verified and approved? Sy whotn? 

Ml *S tvh^t AFSC-^uni^ue or work center-uni<jue traininj 

icquircnients ate not satisfied by the present OJT system? 
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X 



ftH t'tii^Os* etc*) 



, jnit)7 (Obtain 



VI M2.3 X^o is responsible lor the scheduling function 
' inwolwed in it? 



and how (nany others are 



X M2.4 HOW frequently are schedules generated? 



M2,5 Ars there any 



automated aids to scheduling available? 



LL 



M2,6 What are the tra 



tning events scheduled? (Obtain listings) 
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ML. 7 ^lut the i>LobIctn%i t uLuuitt Iri i^clicduling ±jt;inil4(d OJT 4rid 

;incilldry training dCtiviii^ft? Non-i^tdndard <»ctlvitx«9? 



M2.& DO problems exist relative to the quota control and scheduling c>£ 
F^D courses? What are they? 



X 



How do trainees acknowledge the proposed schedule? 



H2.10 I^fi the training schedule have to be coin|>atible with the flight 
schedule? 



X. X rt2-ll What other critical events ^aust be coinpatible with the training 
schedule? 
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1^ r^^ .J*. 



rt7,l2 How arc soijsiotis scheduled fori 
3 * Knowledge (Classroom) 

Performance 
(Draw flow as required) 



y y H2.13 HOW are trainees shcedulea for multi-man task performance trainii^ 
^^^^ and testing? {Draw flow as required) 
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How is testing scheduled loti 

I V I V I 

a. Knowledge? 

b. Prof icioncv? 
(Dtdw flow) 



Vdhat 



t feedback doefi the trainee rece 



Ive a£t«c proficiency testing? 



f...b3c. HC S.p«.v»oc .«.ive ,ft« proficiency t„tin,7 



K2.16 How l£ 



K2.17 What are the 
workcenter? 



is scheduling «c««plish.d for COC CE «dminietr«io«7 



procedures for providing CE revi.w tcaining in this 



MJ.U HOW is CDC CE review training scheduled? 



JJ 
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(MJ) Tidinoe Mjaa<3eiTn?nt Sub-j/stem - Individual Training Rijcord*. 



h2wI What are &quadron/uni t requirements for ttcicking individual 
trainee progress in OJT? 



M3*2 What tracking aids (records Or charts) are employed by you to 

check On ptogress of trainees toward task proficiency? (Obtain 
copies of all lecords i forms utilized) 



f43.6 Should A permdnent^ individual training tecord be initiated aftex 
Basic Military Training? 



t^ow often is each individual training record (623) accessed by you7 



H3.B For what reasons do you access the individual training record (623)7 



M3.9 What elements of the AF Form 623 do /ou single out for extra 
attention when you examine it7 



M3.10 Who else needs access to an individual training record (623)7 



M3.11 Is the information requited from An access of the Individual 
ttairking record (623) desited immediately? 
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wh4t infofmdtion currirntly collected in thft b21 not needed? 



What information needs to be collected that is not pre»(>ntly 
available in the 639? 



M3.14 Ho^ long should information collected in the individu4l training 
recotd he retained? 



H9.16 What cecvrity procedures must be followed in the maintenance of the 
individual training record? 



M3.17 Do.yott maintain a csu *nd do they have an <^'^ Program? 
(Outline features) 
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i^A) Tr^Ainee Management Subsystem - Unit TtAining records 



H4.1 What levels of management require aggregate unit trainin9 data? 



¥iAm2 What levels of management {by organization component) how rfrceii 
aggregate unit training data? 



H4*2 Mtiat aggregate unit training and unit task proficiency/verification 
data is required to afisees trainin9 responsiveness to mission 
requirements? 



For what purpose is aggregate unit trainiftg and unit tasK proficiency/ 
verification data > j » 0 ¥ <Cgw*<^? 



H4.5 What source(s) currently produce unit training data? 



M4>6 What Bhould be the relAtionship between tba unit training cacotd and 
th« KK graduate evaluation progran? 
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M4.7 What oie yoo» petctptio 
ptovtded and its cott? 



m of the value of the t£«tnlng "ow being 



H* 9 What ts your perception of the need to suniMtlxe projection, of 
uaining^ bTaccLpll.hed in the futut.7 



IS there -ny value to be g-ined by producing -t.tlstU. related to 
maNiiDum tt^intng load? 

Utilisation? 



Capacity? 
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(MS) 



Trainee Management Subsystem - Interface* CtJC, HHICS, PDS 



M5,l IB ADP support now provided for any functions or aspect* of the OJT 
program? 

If ye«, what level, of suppott ate provided? (specific.) 
Foe which poBition. i. the support provided? 



MS i What training data exists on PDS (Obtain listings) which «e 
particularly useful to you? 



MS.S What inputs to ADP .ycteM are made by you? 

What are the difficulties in making these inputs? 
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7 



M6.2 TO what degtM U .quipment .v.il-ble for proficiency tt.inin, 
«nd testing? Iteiw in «hott •upply? 



uii 4 How are tcainitM facilities, equipment, filM and •pecisl 

^tructors f« OJT and .ncillary training .ch^duled .nd provid«J7 



H6.t Who naintain. the technical libt.ty in thi. unit? 
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XX 



(H-Alt Skill-level Indices 



H-M.l How is vorkcenter missicm capability defined with respect to skill 
levels? 



What, in your opinion, is the present relationship between skill 
levels and task proficiency? 



H-A1.3 Should skill levels reflect only proficiency in presently assigned 
position? 



H-A1.4 Shoold a ikeans be devised for deriving an indicator of skill level 
based on the niunber and type of positions beld within the specialty 
area, number and types of tasks perforned, and the level of 
proficiency demonstrated? 



M-A1.5 What factors do you think should be considered defining 
skill levels? le*g», task proficiency, task knowledge, 
career knowledge, training, etc.) 
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t£l) Evaluation 6 Training Analysis Subsystem - Individual perforinance 
evaluation 



El.l Hovi Is task proficiency evaluation conducted In this squadron/ unit/ 
wrkcenter 7 



El.l Do proficiency tests require that all trainees perforn all aspects of 
ntulti-^man tasks? How is this accomplished? 



El. 3 How would you respond tQ the suggestion that jPCtaSk performance should 
be tested periodically after up9rsdlf>9? 



£1.4 What Impact does the Maintenance Standards Evaluation Program (hSEP) 
have on the OJT program in this career field? 
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El.S Who fihojld certify the completion of tA«kB Assigned to a 
ttainee? 



El. 6 How &hojld ta»k cectif ictttion be accomplished? 



El. 7 Should there be a base-level quality control function for 
•11 OJT task evaluationc? 
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\L2\ Evaluation 4 Training Analysis Subsystem * Co»t and capacit]^ anal^^sift 



E2.1 Do you find that your normal production duties in the workcenter 

adversely affect the functions expected of you in the OJT program? 
Where are the cooflict«? 



E2-2 What problems have you had relative to OJT capability and capacity? 



E2.Sa. Where do you believe that 93 presently structured is cost 
effective? 



b*ilot cost effective? 
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Evaluation k Training Analysis Subsystes * Intftrface^ MAJCOM 
« Air Staff Training 

E3.1 Have HAJCOH OJT Staff and other training opBs been kept adequately 
informed of problems within the OJT syfitem? 



E3*2 What atea&t if any^ require vto^e definitive reporting? 



E3*3 What additional reports* statisticst and/ar revised reporting 
procedures would b« desirable for HO OJt staff 9nd training 
OpRs? 



&3.4 What measures which may be related to OJT are used by higher 
authority to evaluate your unit's performance? 



E3*S Is Chere a MAJCOM incentiveaward progran in effect for OJT 
trainees and/or crainers? If sot whac are Che mechanics? 
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tEl) Evaluation ( Training Analysis Sabsystem 
EvalaAtion (miesion c«quiceinent0) 



Training £f f ect ivenesB 



E4.1 Wh-l moasutcs ate utiU«d by B^„»dton ^^J^"!"?^" * '"^.ti'IlV' 
■esBss tl>c •ft*Ctiv.>*M «nJ •tOotfiwy t>t aiT in ynut unltif 



E4 2 What additiOMl t.potts «nd/ot tepotting ptoc«l«« «ould be 
desicabl« fot yog in the OJT ptogtM? 



*y «h.t «eans do you keep CBPO C«T *d«inist«tor. 

inforinea of ttaineea* progtess in WT7 
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XX 



eEf«ctiv«ne8« »nd eEIiciency ot OJt? 



« 5 What .t.ti.tic« would r.Elect <«c«r-t.ly th. degree to vhic. 
CUT was meeting mission tequwements? 



E4 



.6 What statistics do yoo cons 



idet significant in evaluatii*9 CWT7 



e4 7 Have yco been »:.pt .6e,..tely inJor«d oE problem within the OJT 
syste™? Wh«e are the aeEiciencies? 
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(Fl) Personnel Subsyfitem » Organizational and peraonn^l requirements 



Pl.l What ar« your perceptions on the concept of consolidation of trainir 
managers at che t>ase level toj 

4. Obtain bettet utilization of *7&' personnel? 



bi. Increase training event scheduling effectiveness? 



Cp eliminate fragmented management? 



d. Improve capability for performing 3rd-pArty taak proficiency 



F1.2 Do you have adequate tine to do vhat is expected of you in your 
role withlni 

The upgrade training portion of OJT? 



Qualification trainiA9? 



Pl,3 Could OJT managers be reorganized in any other manner to improve 
their ability to manage the OJT program? 



evaluation? 



Other? 
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ten Compot<(r Subiyatem - CPU Support 

Ci.i What function. fy-*t.n*J supported by th* ba.-l.vel iy^te^. 



d 2 What training function. «e currently i^^PP^rtea by the baie-level 



CI. 3 Current base level iyeten* 
Nave? 

Hdmory? 

Teroinalfl? 

I/O devices? 

Secondary stotage? 

Operating Syetew? 
PrograiBiing LanguageteJ? 



5p«ciil Purpose languegeli)? 
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CI. 4 Whtt transportability f«aturt» txiat in tht prtotnt bftu*lev«l aystta 
for units »«nt to rtn^tt «ita«7 



Cl.S What type Oif aysttotft) ift/are currtntly in plact vhich could be 
istllize4 for ITS? 



CI. 6 Can a distributtd oonputer syaten be is«ea effectively in the OJT 
environnent? 



CI. 7 What security faattsreB * . incorporated in the current base*level 
fiystett for renote teralail acce»7 
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(C2l CoiKputet Support Sub»y»te» - Coiwuni cation* 

C2.1 Wh.t c«np«ft intetfac.. «xi.t with other C««««dB/b.M.7 



C2.2 Wh«t is the C«tent «P«ity for tel.c«»««i«tioM on the 



baa*- level 
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(C5) Cto»put*r Support Sub^y^e. - Application Pr09rans 

C5 1 Are ,11 proc«Jur« «««>loy«J lit antn autowtlon c^nttrs covered 1-t the 
MlL-STO-490 AFR 300*127 



C5.2 Hh.t for«. exUt (or o.e In «y.t.«. doca»e„t«lon «^ .t-n6.rdi«tlon7 
(e.9.. Ar2053» WJOS*. etej 
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APPENDIX B 



ITS COMPUTER SUPPORT TRADE STUDIES ANALYSIS 



Background 

The purpose of the trade studies phase of this effort was to 
provide an analysis of the cost alternatlveb available for meeting 
\rarlous functional requirements, particularly those for computer support 
of Air Force OJT. The cost data produced were utilized In cbe trade 
studies analysis of the ITS alternative design options. 

Alternative Computer Design Options for ITS 

The requirements In Table 4 of the body of this report present some 
unique system design problems for the development of an Integrated 
Training System for Air Force OJT. The hardware/software options 
available are so extensive that It would be virtually Impossible Co 
Identify three or four meaningful total system design concepts for Trade 
Studies Analysis. Preliminary analysis Indicated that perhaps the most 
feasl&le approach to designing alternatives for the ITS would be to 
concentrate on smaller, Independent aspects of the system. These could 
then be presented In a manner that would allow the Air Force to utilize 
a "mix and match'' method for selecting the most promising options for 
the development of the trade studies analysis as well as for development 
of system specifications In the final system definition phase of this 
effort. This appendix describes alternative design options that were 
considered for: 

1. Configuring the ITS computer network 

2. Collecting detailed task training data for trainee management 
and evaluation. 

XTS/PDS Considerations . Some very Interesting parallel 
relationships exist between several of the functions to be performed In 
the ITS and some of the functions currently performed In the Air Force 
Personnel Data System (PDS). These relationships are Illustrated In 
Figure B-1 , which depicts many of the commonalities that exist between 
the two sysitems. 

1. A requirement exists (see Table 4, Fl) to provide a means to 
establish and maintain for each airman an Airman Training Record that 
would contain a history of training completed during the airman* s career 
and that would be forwarded to each base of assignment. A small amount 
of training data Is currently maintained In PDS, and these functions In 
ITS could be accomplished In the same manner as maintenance and 
forwarding of personnel data In PDS. 

2. A requirement exists (see Table 4, F6) to collect and maintain 
Information related to the personal attributes of each airman. This 
same type of data Is also maintained In PDS. 
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Figure B-K ITS/PDS Relationship 
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3. Other requirements are (See Table 4, 12, L9, M3, M5, M6, M8, 
M9, MIO, and Mil) to provide analyses of ITS data and produce summary 
reports of unit training events and the effectiveness and efficiency of 
OJT to the MAJCOM/HQ USAF. PDS accomplishes this type of function In 
Its three^tler vertical data flow concept. 

A. A requirement Is also defined (See Table A, N3) to provide 
Information to the PDS related to an airman's current position 
qualification and his/her current AFS development to support promotion, 
assignment, classification, find other career development objectives. 

Discussions with AFMPC/MPCD data automation personnel during the 
data gathering phase of this effort Indicated that It Is conceptually 
feasible to consider designing the airman training record portion of the 
ITS system utilizing the PDS software system In order to take advantage 
of the data base management and horizontal/vertical data flow concepts 
already established. It would also be feasible to establish an 
Interface between ITS and PDS through the PBS buffer transfer concept to 
accoupllsh updates of common data elements. Because of these factors, 
all ITS computer support alternatives consider the ITS airman training 
record as an Integral part of PDS. 

ITS Network Configuration Alternatives . The computer/data commu- 
nications network options available to support a system of the magnitude 
projected for ITS are extensive. There are many different combinations 
of hardware/ communications networks which potentially satisfy these 
requirements with the specific design dependent on the unique capabili- 
ties of each prospective vendor. The various alternatives applicable to 
the projected ITS requirements can be generally described In terms of 
(a) a dedicated large-scale centralized system, (b) a distributed, 
dedicated network of mini-computers, (c) a major, full"scale 
augmentation of the planned Phase IV base-level system to Include ITS, 
or (d) a relatively minor auga' ntatlon of the planned Phase IV base- 
level system to support some components of ITS, with the remaining 
components supported by dedicated micro-processors and/or mini" 
computers. Each of these alternatives would Include a Trainee 
Management Support System (TMSS) component. TMSS alternatives are 
described In the ITS Trainee Management Support Alternatives section of 
this appendix. 

a. ITS Network Configuration Alternative A - Dedicated Large Scale 
System . This alternative, as depicted In Figure B-2, would provide a 
large-scale, multi-processing computer system at a single central site. 
Access to the computer from each base would be provided via a network of 
dedicated telecommunications lines Interfaced with a message processor 
at each base. All ITS processing and updating of files would be accom^ 
pushed at the central site. Interfaces with MAJCOMs, HQ USAF, and the 
PBS would be via AUTODIN, dedicated line, or mall from the central site. 
Support for the special base-level trainee management aspects of job- 
utte tr^ilnlng would be provided via nilcro*processor or small mlnl- 
computerj} Interfaced with the message processor. 
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The advantages of this alternative relate to benefits that are 
typically present In any large-scale, centralized computer system. Data 
base design and efficiency are easier to control. Software changes can 
be effected In a more timely manner. A typical large-scale system would 
provide for greater processing power In statistical analysis, simula- 
tion, modeling, etc. The cost of processing each transaction Is usually 
relatively low. 

From a negative standpoint, this alternative would require 
extensive use of data communications lines which have traditionally been 
a major weakness In Interactive systems. Communicating the status of 
the system or problems with the system to the users Is extremely 
difficult and when the system Is down, all users are down. Access to, 
and security of, the data base Is more difficult to control. Perhaps 
due to the remoteness of their access, users often perceive that their 
priority In the use of the system Is at the bottom of the scale. There 
are also limits to the amount of customizing that can be accomplished to 
support the unique requirements of each base* Control of computer 
resources would not be closely aligned with the ITS management 
organization at each base. 

b. ITS Network Configuration Alternative B - Distributed Mini- 
Computer System . This alternative, as depicted In Figure B-S, would 
Interface via telecommunications lines a series of mini-computers 
located at each Air Force base to a small-scale computer system (or 
mini-computer) acting as a central repository ""hub** for historical 
training data bases, and as a clearing house for Inter-base transfers of 
ITS records and Interfaces with other systems. Each base-level ITS 
system would have a complete set of ITS records and files pertaining to 
Its AFSCs and personnel. All processing of Individual task proficiency 
training requirements and updating of ITS files would occur on the base- 
level system. Data required at the central repository would be for- 
warded In batches on a dally basis. The MAJCOMs and HQ USAF would have 
direct link to the central repository of ITS data via telecommunication 
lines and would have access to software designed to analyze data and 
produce the evaluation reports of the system. 

Since most of the computer resources for this alternative would be 
distributed at the base level. It would eliminate the need for an 
extensive data communications network for the critical day~to-day 
operational components of the ITS system. Computer downtime would 
affect only the small segment of users being supported by that system. 
The users would be more likely to view It as their system, and 
therefore, user acceptance would probably be greater. It would also 
permit a closer alignment of ITS computer resources with the ITS 
training management organization at each base. 

Statistical analysis, simulation, and modeling capabilities would 
be minimal, as mini-computers typically do not possess the processing 
power of large~scale systems. Control and dissemination of software 
changes would be more difficult to accomplish. Once the data base for 
the central repository of historical data Is designed. It would be much 
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more difficult to adjust the format to reflect changing requirements at 
each base. This alternative would also result In a less efficient use 
of computer personnel resources and a probable requirement for 
additional computer personnel. The cost to process each transaction In 
a distributed system Is typically higher than for a centralized system. 

c* ITS Network Configuration Alternative C - Major Augmentation of 
Phase IV Base*I>evel System * This alternative^ as depicted In Figure 
B-4> would provide the total base-level support for the requirements of 
ITS by augmenting the capacity of the planned Phase IV base-^level 
system. The functions of ITS to be supported by the **hub'* system would 
be provided by either the PDS computer system at AFMPC or a dedicated 
sinall**scale system Installed at a designated site. 

This alternative offers the same advantages as described for the 
distributed mlnl^-computer (Alternative B) » but In addition^ would 
provide a direct Interface and link with the PDS system. There are also 
potential cost savings In using the PDS software system In lieu of 
designing new software for maintenance of the airman training record and 
to satisfy th^ horizontal/vertical flow requirements of ITS. Since the 
Phase IV system would already be In place^ personnel would be familiar 
with the computer equipment and terminals^ and control of the additional 
computer resources required by ITS could be merged Into the existing Air 
Force Data Processing Installation (DPI) organization. 

This alternative offers the same disadvantages as described for the 
distributed mini-* computer concept (Alternative B) . Additionally^ If the 
Phase IV system continues to use AUTODIN for the transfer of data> It 
could be considered a serious disadvantage In that use of AUTODIN Is 
slow^ cumbersome^ and sometimes unreliable. The requirements of ITS 
could possibly exceed the capability/capacity of the Phase IV computer 
system at each base> thus necessitating another procurement for the 
base-level systems. 

d. ITS Network Configuration Alternative D ^ Minor Augmentation of 
Phase IV Base^Level System * This alternative^ as depicted In Figure 
B-3^ would provide support for the requirements of ITS by removing the 
trainee management support functions from the Phase IV base-'level system 
thus resulting In a significantly $caled~down augmentation of the base-* 
level system. In addition^ a network of mlcro-^processors or mini- 
computers would be provided to support the trainee management functions 
of ITS at each base and these would be Interfaced with the Phase IV 
system through a message processor or by multiplexing several units to a 
dedicated communications line on each base. 

This alternative offers the same advantages outlined In Alterna- 
tives B and C. In addition^ this alternative probably eliminates the 
possibility of exceeding the capability/capacity of the Phase IV 
computer system cited as a potential disadvantage In Alternative C. 
With that exception^ this alternative offers the same disadvantages 
outlined In Alternatives B and C. 
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Figure B-5. Network Conf iquratlon Alternative D ~ ^^."^ Au<3m«ntation of Phase IV Base Level System 
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ITS Trslnee Management Support Alternatives * When the system 
requirements In Table 4 were examined In depth, the trainee management 
functions Identified In the Instructional management category were seen 
to be critical to the success of the system and to represent a most 
significant Improvement over those of the current OJT program. Most of 
the Instructional management requirements Identified In Table 4 have 
been reflected In a general procedure flowchart In Figure B'6 that 
depicts the major steps required In managing the training of a typical 
trainee In a work center. These steps are distributed across the 
following training processes: the OJT orientation and diagnosis of 
training process, the job task training management process, the task 
proficiency training management process, and the quality control task 
evaluation process. Figure B-6 Is Intended to Illustrate a general 
outline for the OJT trainee management process; It Is not all'lncluslve 
and should not be used In lieu of the requirements outlined In Table 4. 

The quantity and currency of tralnlng*related data that will be 
collected from the trainee, trainer and supervisor at various points In 
the task training process, coupled with the Input frequency required by 
the management controls specified for task proficiency training will 
necessitate that a cost-^ef f ectlve. Interactive means be provided for 
data entry. Several state'^of'the^art possibilities currently exist 
which could satisfy the trainee management data collection requirements 
of the ITS with varying levels of human effort Involved. Each of the 
electronic alternatives was considered plug'to'plug compatible with most 
computer systems and could be compared Independently of the basic 
computer system supporting the ITS. Because of this, the Air Force 
could select more than one type of "trainee management support system" 
If required to achieve the maximum degree of flexibility In supporting 
various operational requirements within work centers/sections. 

The following alternatives were considered to be conceptually 
feasible options for the ITS In support of job task proficiency 
training. Each alternative provides a means to record the action 
accomplished, the trainee affected, the trainer performing the action, 
supervisor decision actions, and any other task proficiency training 
data required for operation of the system. 

a. ITS Trainee Management Support System Alternative A - Portable 
Data Terminal Using Bar Codes and/or Optical Character Reader (OCR) 
Format . This method would use an optical scanning wand connected to a 
portable (hand*held) data terminal. Plastic tabs or Btlck*on labels 
would be produced for each action. Individual, and task In the work 
center/section. These would then be Inserted on a large board or on job 
performance training aids In the work area for use by all concerned. 
The required data would be entered Into the portable device by passing 
the wand over the applicable bar code In the sequence specified for that 
action. 

The portable data terminal would then be connected to a data 
communication line at specified times or randomly throughout the day to 
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Figure B-6. Procedures for Management of Task Proficiency Training 
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Figure B-6. Procedures for Management of Task Proficiency Training (Cont'd) 
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Figure B-6. Procedures for Manaqement of Task Proficiency Training (Cont'd) 
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transmit the data collected to the computer system for updating the 
Individual training record. 

b. ITS Trainee Management Support System Alternative B - Magnetic 
Card Concept . This method would employ magnetic cards similar to those 
used by banking teller machines or the Washington, D.C., Metro transit 
system. 

1. Option 41 . As 3oon as the diagnosis of training 
requirements process Is completed, a magnetic card containing the 
trainer ID, task number, task training steps, and other data would be 
generated for each job task to be trained. This set of cards would be 
stored In a pigeonhole designated for the trainee. Similar cards would 
be available to Identify trainers and supervisors If required by the 
action being recorded. At each data point In the training process for a 
job task, the appropriate card would be removed from the pigeonhole and 
inserted Into a reader device that Is connected to a mlcro*-processor or 
computer. Function keys would be pressed to Indicate the action being 
recorded, the magnetic card itself would be ipdated, and a record of the 
action would be forwarded by the micro-processor to the computer for 
updating the Individual training record. 

2. Option tf2 . This method Is similar to Option 1 except that 
the U.S. Armed Forces ID card would be used to enter ID data and the job 
task trailing procedural data would be prerecorded on a job performance 
training aid. At each data point In the training process for a job 
task, the ID card(s) and the job performance training aid would be 
Inserted In a reader device. Function keys would be pressed to Indicate 
the action being recorded and the appropriate data would be forwarded by 
the mlcro^processor to the computer for updating the Individual training 
record, 

3. Option i^3 . This option Is the same as Option 2 except that 
a prerecorded magnetic card would be used In lieu of the job performance 
training aid for Identification of the task training requirements and to 
monitor the correct sequence of steps In the training process. 

c. ITS Trainee Management Support System Alternative C " CRT 
Concept , This method would utilize a cathode ray tube (CRT) with a 
keyboard and a direct Interface with the computer to enter data Into the 
system. A series of screens In menu format would be utilized to guide 
the user through a complete transaction. All data would be keyed Into 
the CRT device either through direct keying or by bulk selection via the 
menu process . 

1. Option fil . This method could be expanded to support the 
task knowledge testing process as well as the delivery of selected 
instructional materials In a CAI mode, 

2. Option 02 m This method could be combined with the magnetic 
card concept described previously to replace/augment many of the menu 
format screens. 
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3* Option tf3 * This method could be combined with the portable 
data terminal concept described previously to facilitate the data entry 
requirements and allow data collection at the training/work location. 

4. Option il^A * The direct Interface with the computer could be 
replaced by using a small stands-alone system In which the ITS and data 
updates would be passed via diskettes through the base messenger ser- 
vice. 

5. Option #5 * This method would utilize a touchrpanel display 
device such as the plasma terminal or an optical light pen option on a 
CRT to allow the user to touch the screen for Identifying data to be 
recorded In lieu of keying In data to be recorded. 

d. ITS Trainee Management Support System Alternative D - Polnfof* " 
Sale (POS) Keyboard Concept > This method would utilize the polnt-of- 
sale<-type or POS-type devices that are now common In cafeterias » fast 
food franchises^ and other retail locations. A polnt<-of-sale keyboard 
consisting of 150 to 200 keys would be coded or designated with 
descriptive statements Indicating the function of each key. Under this 
concept, a key would be reserved for each person In the work center and 
for each job task performed In the work center* Some keys could also be 
reserved for special functions such as action codes and numeric data. 
This device could also be equipped with a small printer to print an 
abbreviated record of the transaction as well as some Information for 
the trainer and/or trainee. 

At each step In the training process, a designated person would 
press the action code key. The device would provide prompts through a 
sequence of steps In which the keys Identifying the trainee, task, and 
trainer were pressed. When the data entry was complete, the data col<- 
lected would be transmitted by the POS device to the computer for up- 
dating the Individual training record. 

e. ITS Trainee Management Support System Alternative E - Audio 
Input Concept . This method would utilize a device designed to convert 
the spoken word of the human voice Into electronic digital 
representation so that It can be recorded /processed In a computer 
system. The current technology of audio Input devices Is such that 
portable micro-recorders could be taken to the work area when training 
Is being accomplished and the trainer could Indicate each action 
accomplished, by making simple oral statements. At a convenient time, 
the micro-recorder would be attached to the audio Input device and the 
statements would be converted and transmitted to the computer system for 
updating the Individual training record. 

f . ITS Trainee Management Support System Alternative F - Coding 
Form Concept . This method would utilize coding forms designed to enter 
all of the data required for each action. These forms would be 
collected dally and read by an optical scanning machine linked to the 
computer or forwarded to a data entry office which would convert the 
coded data Into a machine-readable form. Recent technological 
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advancements In the microform field would allow a microform output 
option to be combined with coded Input to eliminate any direct 
requirement for computer support In the work center/section. 

Cost Analysis of ITS Computer Support Requirements 

The computer support requirements for ITS were categorized Into 
three different levels for comparative purposes. Processing require- 
ments were defined for Air Force-wide functions, base-level functions, 
and trainee management support functions; each of these was considered 
to be Independent of the other In terms of design options and projected 
costs for the system. Costs estimated In all alternatives were 
developed by averaging data obtained from Industry and Government 
sources . 



ITS Trainee Management Support System (TMSS) Alternatives Coat 
Analysis . The objective of this section is to analyze the alternative 
options available for computer equipment designed for directly 
supporting the task training process at the work center level. Each of 
the TMSS alternatives described In the previous section was analyzed for 
technical feasibility with the ITS design alternatives presented 
elsewhere In this report. The feasibility of each TMSS alternative Is 
Indicated In Table B^l, with a "Ho" Indicating that either the ITS 
alternative does not support the requirement for use of that TMSS 
alternative or that the TMSS alternative Is not a feasible approach to 
the requirement. 



TABLE B-1. FEASIBILITY OF TMSS ALTERNATIVES 



ITS Alternative 





TMSS Alternative 


1 


2 


3 


A. 


CRT/Portable Data Terminal 


Ho 


No 


Tes 


B. 


CRT/Magnetic Card 


No 


No 


Yes 


C. 


CRT/Prlnter 


Yes 


Yes 


Yes 


D. 


CRT/Point of Sale 


No 


Yes 


Yes 


E. 


Audio Input 


* 


* 


* 


F. 


Coding Form 


Yes 


No 


No 



* Technology promising but not considered In cost analysis due to 
limited availability of data. 
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In each of the TMSS alter^atlved, It wad envisioned that a variety 
of computer equipment devices vould be required to provide an effective 
and functional environment for the ITS user. The projected badlc 
equipment requlrementa to support a typical base for each TMSS 
alternative are provided In Table B-2. 



TABLE PROJECTED TMSS EQUIPMENT REQUIREMENTS 



ITS Alternative 



Equipment 


1 


2 




3 








TMSS Alt. 
C 


TMSS Alt. 
C D 


A 


TMSS 
B 


Alt. 

C 


D 


CRT 


5 


20 20 


75 


75 


75 


20 


Printer 


5 


20 


75 




75 


20 


Optical Scanner 




20 20 










Portable Data 
Entry Device 






75 








Magnetic Card System 








75 






Point of Sale 




20 








75 



Cost data were developed for the options Identified In Table B'~l 
and B-2 that were determined to be feasible In satisfying the require- 
ments for each alternative configuration for ITS. The number of ITS 
support stations for each ITS alternative Is specified In Table B-3. An 
ITS support station wad defined as a location where automated computer 
support related to trainee management iiuidt be provided for data collec 
tlon/report generation In direct support of a trainee, trainer, or 
supervisor. The cost of each TMSS option and the delected option for 
the cost analysis of each ITS alternative are provided In Table B'-3. It 
should be noted that the decision to eliminate or discard any one of 
these options based on cost alone would not be appropriate until the 
potential benefits of each are explored In a demonstration under a 
variety of job-site training conditions. 
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TABtE COST ANALYSIS OF TMSS ALTERNATIVES 

($ per Typical Base) 



ITS Alternative 





TMSS Alternative 


1 


2 


3 




Number of Base-Level ITS 
support &caciona 


5 


20 


75 


A 


wl\ I / JrOrt3DXe va^a IdUllTlaX 






397 305 


B. 


CRT/Magnetic Card 






892,160 


C. 


CRT/Prlnter 


38,470* 


452,984* 


337,305* 


D. 


CRT/Point of Sale 




484,244 


418,758 


E. 


Audio Input 








F. 


Coding Form 


** 







* Selected alternative for ITS Trade Studies Analysis 
** Combined with ITS Alternative 1, Option C, for the cost analysis 



ITS Base-Level/Alr Force^Wlde Coaputer Support System Cost 
Analysis . The objective of this section Is to provide an analysis of 
the computer hardware options available for linking/interfacing the 
training environment and each TMSS with the computer providing support 
for the processing/reporting functions of ITS. The degree of computer 
support that would be provided at the base level Is totally dependent on 
the major procesalng functions to be performed In relation to the 
network configuration alternative selected. In system design 
Alternative 1» network configuration alternative D was not costed» as 
the processing requirements did not generate the need for an ITS mini- 
computer. In system design Alternative 3, network configuration 
alternative C was determined to be not operationally feasible, due to 
the projected workload requirements of OJT. 

Table B-4 contains a summary of the discounted life cycle costs of 
computer support options for the three levels of ITS support required 
for each ITS design alternative. These cost figures Include estlmatea 
of the computer equipment costs, network telecommunications costs, and 
computer support personnel costs for each of the network configuration 
alternatives for both the base"level support required and for the Air 
Force-wide support of OJT. Detailed cost data associated with the 
options Identified by an asterisk (*) In Table B"4 were utilized to 
compute the total estimated cost of each ITS dealgn alternative. 
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TABLE COST ANALYSIS OF ITS COMPUTER SUPPORT SYSTEMS 

(lO-year discounted life cycle costs) 
( In millions) 



ITS Alternative 



Network Configuration Alternative 


1 


2 


3 


A. ITS Central System 


41.74 


107.04 


164.57 


B. ITS Distributed System 


143.86 


205.12 


387.57 


C. AP Phase IV System 


20.27* 


96.10* 




n. AP Phase IV/lTS Mini 




105.20 


117.12* 



* Selected alternative for ITS Trade Studies Analysis 



Demonstration Considerations . The development and demonstration 
phase of the ITS presents some Interesting considerations with regard to 
the computer hardware requirements to support the effort. An analysis 
was also performed during the ITS computer trade studies analysis to 
determine the feasibility of using the CYBER 73-16 computer facility at 
Lowry APB to support the demonstration, as opposed to the system 
Identified to satisfy the requirements of the ITS preferred alternative. 
An analysis of this ITS development approach is provided In Table B"5, 
with the conclusion th*:t the advantages of using the CYBER 73-16 for the 
development and demonstration of ITS would outweigh the disadvantages as 
long as adequate controls can be established on software development to 
minimize the reprogrammlng and transition costs Incident to later full-^ 
scale Implementation. 
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TABLE B-5. 


ITS DEVELOPMENT/DEMONSTRATION CONSIDERATIONS 
COMPARISON OF CYBER 73-16 VS PHASE IV/lTS MINI 








CYBER 73-16 


ITS Alternative 3 
Phase IV /ITS Mini 




Performance 
Advantages 


o o o o o o 


Adequate for DDT&E 
Available 

Specialized device for 
DDT&E 

Use of AIS for Instruc- 
tional/Trainee Manage*" 
ment functions 
Could become repository 
for job task Identifica- 
tion functions 
Could become retiosl t orv 
for training development 
functions 


o Adequate for DDT&E 
o Will be more like 
operational system 




Disadvantages 


o 
o 


Could require extensive 
redesign of software at 
Implementat Ion 
Conversion of PDS soft- 
ware for ATR functions 


o May take time to acquire 

approval 
o Possible disruption to 

other base-level users 
o Uncertainty of Phase IV 

delivery 




Logistics 
Advantages 


o 


Use of R&D Resources 


o Compatible with PDS 
o Conversion problems 
minimized 




Disadvantages 


o 


Incompatible with PDS 


o Planning and coordina- 
tion with Phase IV 
management and base- 
level DPI 




Cost 

Advantages 


o 


Minimal cost 


o Software development 




Disadvantages 


o 
o 


Possible telecommunica- 
tions cost to demonstra- 
tion 

Software conversion at 
Imp lemen t a t Ion 


o Cost of Initial equip- 
ment 
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GLOSSARY OF TERMS 

Air Force Specialty (AFS) . A group of positions that require common 
qualifications. Each Air Force specialty has a title and Is Identified 
by a numeric code. 

Air Force Specialty Code (AFSC) . A combination of alpha and numeric 
characters used to Identify an Air Force specialty. 

Airman Training Record (ATR) . A conceptual ITS data base that contains 
an historical account of all training completea' by an airman throughout 
his/her Air Force career. 

Base~Level Training Requirements (BLTR) . A conceptual ITS data base 
that Identifies those training requirements which airmen must complete 
that are not related to an Air Force Specialty. 

Career Development Course Requirements (CDCR) * A conceptual ITS data 
base that Identifies requirements and controls for each career develops 
ment course used within the Air Force to satisfy career knowledge 
prerequisites for award of an Air Force Specialty Code skill level. 

Comprehensive Occupational Data Analysis Programs (CODAP) . A set of 
computer programs used to automate, process, organize, and report 
occupational/task data* 

Consolidated Base Training Office (CBTO) . A conceptual agency for the 
ITS given the responsibility for scheduling, managing, and evaluating 
all training conducted at a base. 

Course Training Standard (CTS) . A course control document. It 
prescribes the qualitative requirements of a formal course In terms of 
tasks, knowledge, and proficiency levels (extent of training). 

Generic Position Training Requirements (GPTR) . A list of tasks, drawn 
from the master task list, representing a generic duty position within 
an Air Force specialty. Capable of being transformed Into an 
operational PTR listing tasks actually performed In a specific duty 
position. 

Individual Training Requirements (ITR) . A conceptual ITS data base that 
Identifies and provides status of specific job task training require-^ 
ments for an airman In his/her assigned duty position. It Is envisioned 
as an Interim record which augments the airman training record to 
provide a complete training record of both the training In progress and 
training completed for the airman. 
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Instructional Management/Delivery Control (IMDC) . A conceptual ITS data 
base that provides the linkage to the Instructional management and 
delivery specifications established for the task In the HIKDC. It would 
be utilized to direct and control learning activities of airmen and 
provide management Information to their supervisors and trainers during 
the training process. 

Instructional System Development (ISP) . The five-step, systematic 
approach for developing Instructional systems. 

Integrated Training System (ITS) . A computer-based system for the 
development, management, and quality control of Air Force OJX. 

ITS Support Station . A location where automated computer support 
related to trainee management must be provided for data 
collection/report generation In direct support of a trainee, trainer, or 
supervisor In the OJT process. 

Job Proficiency Guide (JPG) . A document containing Air Force Specialty 
Code*related task descriptions used by Air Force supervisors as a guide 
for conducting training at a job site (now JQS, Job Qualification 
Standard) . 

Maintenance Management Information and Control System (MMICS) . The 
automated system used within the Air Force logistics functions to manage 
and control maintenance-related data. Discussions In this paper 
reference the training subsystem of the MMICS. 

Master Instructional Management/Delivery Control (MIMDC) . A conceptual 
ITS data base that will contain standardized detailed descriptions of 
steps In the training process for task proficiency objectives. It would 
Include the controls needed for proper sequencing of training and 
collection of data needed to measure trainee progress and provide 
management Information. This data base would be available throughout 
the Air Force for use by units where Identical tasks are trained. 

Master Task List (MTL) . A conceptual ITS data base that will contain 
all job tasks for each specialty within the Air Force. It would be used 
to standardize the Identification and description of all tasks to 
provide for universal use throughout the Air Force. 

Master Task Training Requirements (MTTR) . A conceptual ITS data base 
that will contain detailed descriptions of job tasks for AFSs, the 
status and progress of development of all Instructional materials and 
related documents, controls, and tools for building the MIMDC data base, 
and a catalog of all Instructional materials related to training of 
tasks. 
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Occupational Survey (OS) . The Air Force procedure for the 
Identification of the duties and tasks which comprise one or more 
shredouts, prefixes, specialties, career field ladders, or utilization 
fields; and for the collection, collation, and analysis of Information 
concerning such duties and tasks. 

Operational Position Training Requirements (OPTR) . A version of an ITS 
generic FTR that has been operatlonallzed by the work center supervisor. 

Personnel Data System (PDS) . The automated system used within the Air 
Force for personnel management at the base and Headquarters Air Force 
levels . 

Position - Standard AFR 35-1 definition! The duties and tasks that make 
up the job or work requirement for one Individual. 

- ITS definition: That group of duties and tasks assigned to 
an Individual airman on a continuing basis. Positions would be 
operatlonallzed within ITS through the OPTR and be Identified by a 
"position number" and/or a "position title*" There may be more than one 
Individual assigned to the same position number, but If one Individual 
has even one duty or responsibility that Is different from the others, 
that Individual should be assigned a different position number/title 
which defines the additional/ different duties and responsibilities* 
This definition does not preclude a supervisor assigning additional 
dutles/responslbllltles to a position on a temporary basis to compensate 
for an unexpected manpower shortage caused by such factors as emergency 
leave. Illness, delays In personnel reporting to the unit, etc* 
However, If the Individual Is performing the additional duties on a 
continuing basis, the position should be redefined* In short, any 
change which occurs In the dutles/responslbllltles performed by an 
Individual on a continuing basis should be accompanied by a change In 
that Individual's position. 

Position Training Requirements (PTR) . A conceptual ITS data base that 
Identifies all positions In a work center and each task applicable to 
each position* It would be used as a basis for Identifying position and 
Individual training requirements* 

Qualification Training . Actual "hands-on" task performance training 
designed to qualify an airman In a specific duty position. This portion 
of the dual'channel OJT program occurs both during and after the upgrade 
training process. It Is designed to provide the performance skills 
required to do nhe job* 

Quality Control (QC) * A conceptual ITS data base that contains a record 
of each job task and the Identification of each airman for whom 
certification to perform the task has been completed. 
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Specialty Training Standard (STS) . A training control document used In 
the standardization and quality control of airman training! Identifies 
general study references and contains a specification of subject 
knowledge levels, task knowledge levels, and task performance levels 
required for each skill level In a specific AFSC. 

Task Proficiency Objective . A specific statement of each task to be 
performed (Including standards of performance) In each authorized 
position In the organization, within which OJT Is being provided. 

Trainee Management Support System (TMSS) . The computer hardware and 
communications equipment at an ITS support station used to enable 
trainees, trainers, and supervisors to Interact with the ITS during the 
task training process. 

Training Guide (TG) . A plan used In the conduct of training. 

Training Resources (TR) . A conceptual ITS data base that contains a 
detailed account of the resources required to conduct job qualification 
training for every position on a base and those required to provide non-* 
job'related training. 

Unit Performance Summary (UPS) . A conceptual ITS data base that 
provides a summary of the task certifications that have occurred and all 
training events completed within a given work center during specified 
time Intervals. This would provide the basis for analyzing training 
effectiveness In relation to mission priorities. 

Upgrade Training (UGT) . That training airmen must receive to qualify 
for award of an AFSC skill level. 
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LIST OF ABBREVIATIONS AM) ACRONYMS 



AF 


Air Force 


AFB 


Air Force Base 


AFHRL 


Air Force Human Resources Laboratory 


AFMPC 


Air Force Manpower and Personnel Center 


AFR 


Air Force Regulation 


AFRES 


Air Force Reserve 


AFS 


Air Force Specialty 


AFSC 


Air Force Specialty Code 




A A^ri^v% ^ j^A Tna^ ^11 ^ A Anal Q v om 

AG van ceo insc^ Lucc^xonsx oy bl^cui 


ANG 


Air National Guard 


APDS 


Advanced Personnel Data System 


APR 


A i rniAn P AT'^oTm ^nr A Report 


ARF 


Air Reaerve Forced 


ASVAB 


Armed Services Vocational Aptitude Battery 


ATC 


Air Training Command 


ATR 


Airman Training Record 


AUTODIN 


Automated Digital Information Network 


BLTR 


Base Level Training Requirements 


BUT 


Basic Military Training 


CAI 


Computer Assisted Instruction 


CBPO 


Consolidated Bade Personnel Office 




uonsoxiGaceo oase i raining vxx ice 


CDC 


Career Development Course 


CDCR 


Career Development Course Requirements 




uivix engineering 


CE 


Course Examination 


CODAP 


Comprehensive Occupational Data Analysis Programs 




Lentrax rrocessing unic 


CRT 


Cathode Ray Tube 


CTS 


Course Training Standard 


DDT&E 


Design » Development » Tedt» and Evaluation 


npT 




ECI 


Extension Course Institute 


ESBI 


Educational Subject Block Index 


FAA 


Federal Aviation Administration 


FTD 


Field Training Detachment 


GPTR 


Generic Position Training Requirements 


HAF 


Headquarters Air Force 


HQ 


Headquarters 
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ID 


Identification 


IG 


Inspector General 


IHD 


Instructional Management /Delivery 


IMDC 


Instructional Management/Delivery Control 


ISD 


Instructional System Development 


ITR 


Individual Training Requirements 


ITS 


Integrated Training System 


JrG 


Job Froflclency Guide ^now JQ5^ Job Qualification Standard) 


JrTT 


Job Position Technical Training 




LiOgis^ics ^omposi^e node J. 




niiitary Axrilrt C^ominana 


MAJCOM 


Major Command 


MET 


Management Engineering Team 




nas uer i ns urUCuionaj. neinagemenu^ uexivery V/onuro± 


MMICS 


Maintenance Management Information and Control System 


MSEP 


Maintenance Standards Evaluation Program 


MTL 


nafiter TasK List 


MTTR 


naster TasK i- raining tvequir emends 


NCO 


Noncommissioned Officer 




Noncommissioned urricer in ^narge 


OCR 


Optical Character Reader 


uJT 


On"'ther'Job Training 




uccupauionai Measurement vender 


OFR 


Office of primary Kesponsl blllty 


OPTR 


Operational Position Training Requirements 


OS 


Occupational Survey 


PDS 


Personnel Data System 


PMEL 


Precision Measuring Equipment Laboratory 


POS 


Point of Sale 


PTR 


Position Training Requirement 


QC 


Cfuallty t^ontrol 




Research and Development 




strategic Axr command 




L CluCfJlC Alt V/O mUlclTlU EVCJJUXO UlUll 


SEI 


Special Experience Identifier 


SKT 


Specialty Knowledge Test 


SME 


Subject-Matter Expert 


SOA 


Separate Operating Agency 


SSAN 


Social Security Account Number 


STS 


Specialty Training Standard 



ERIC 



166 



172 



TAG 


Tactical Air Command 


TC 


Training Guide 


FMSS 


^ 4 ^ ¥4 A a Mart 4 i*t Om 0 t% t~ Qimnn^ ^ Q if q f" Om 


TO 


Technical Order 


IK 


i raining Keaourcea 


UDB 


Unified Data Baae 


UGT 


Upgrade Training 


UPS 


Unit Performance Summary 


VRE 


Volume Review Exerclae 


WARSKIL 


Wartime Skill Program 


W/C 


Work Center 
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